OSCR

Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction.

Code ↔ Paper

40 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

34 blocks of code here have a permanent identifier (oscr:b:…), listed with the matches below: citable for good, with a page of its own. Select a match and press Cite to copy it. What a block identifier is.

The 40 matches
  1. [1] § Methods › Attention extraction and edge score computation ↔ src/05_causal_ablation/21_scgpt_forward_ablation.py, lines 1 to 89 · score 0.99 · F.multi_head_attention_forward, need_weights, bypasses hooks, nn.MultiheadAttention, column slice, proj weight · block oscr:b:GFJ5V1KGAJQV5-G · Report a mismatch
  2. [2] § Results › Causal ablation reveals distributed redundancy ↔ src/05_causal_ablation/20_scgpt_ablation.py, lines 1 to 63 · score 0.99 · Tabula Sapiens tissue, Cross architecture replication, HVG vocabulary, Geneformer head mask, distributed redundancy, forward pass intervention · block oscr:b:PWQJJ9FM3R5M9-U · Report a mismatch
  3. [3] § Results › Causal ablation reveals distributed redundancy ↔ src/05_causal_ablation/21_scgpt_forward_ablation.py, lines 1 to 89 · score 0.98 · Cross architecture replication, proj weight, column slice, TRRUST ranked heads, zeroing head, residual stream · block oscr:b:GFJ5V1KGAJQV5-G · Report a mismatch
  4. [4] § Results › Positive control: the pipeline detects planted regulatory signal ↔ src/13_synthetic_validation/20_positive_control_pipeline.py, lines 498 to 601 · score 0.96 · minimal indirect propagation, strong indirect propagation, weak direct edges, shuffled ground truth, stress variants, planted ground truth · block oscr:b:9H096ACWM36Y9-W · Report a mismatch
  5. [5] § Results › Causal ablation reveals distributed redundancy ↔ src/05_causal_ablation/05_intervention_fidelity.py, lines 1 to 51 · score 0.92 · hidden state cosine, materially perturb internal, ineffective interventions, Intervention fidelity, orthogonal interventions, uniform attention · block oscr:b:AV1ATRMGKMY8C-8 · Report a mismatch
  6. [6] § Results › Context-dependent attention–correlation relationship ↔ src/06_cross_context/08_non_k562_replication.py, lines 1 to 67 · score 0.91 · RPE1 retinal epithelial, derived neurons, K562 replication, iPSC, Cross context replication, evaluable perturbations · block oscr:b:WEF5A62MRXB56-4 · Report a mismatch
  7. [7] § Results › Causal ablation reveals distributed redundancy ↔ src/05_causal_ablation/04_orthogonal_interventions.py, lines 1 to 45 · score 0.91 · orthogonal interventions, destroying attention patterns, uniform attention replacement, zeroing FFN, preserving head, MLP ablation · block oscr:b:1NGJEG5EYXT6C-W · Report a mismatch
  8. [8] § Methods › Model coverage by analysis family ↔ src/05_causal_ablation/20_scgpt_ablation.py, lines 1 to 63 · score 0.91 · Causal ablation evidence, scGPT ablation, forward pass intervention, distributed redundancy, cross architecture, head_mask · block oscr:b:PWQJJ9FM3R5M9-U · Report a mismatch
  9. [9] § Results › Context-dependent attention–correlation relationship ↔ src/15_biological/05_context_dependence.py, lines 47 to 122 · score 0.86 · derived neurons, iPSC, K562 replication, perturbation genes forced, Cross context, retinal · block oscr:b:X1E1H36KHH7RX-5 · Report a mismatch
  10. [10] § Methods › Causal ablation ↔ src/05_causal_ablation/05_intervention_fidelity.py, lines 1 to 51 · score 0.86 · logit cosine distances, Intervention fidelity, Orthogonal interventions, uniform attention, hidden state, ablation · block oscr:b:AV1ATRMGKMY8C-8 · Report a mismatch
  11. [11] § Methods › Causal ablation ↔ src/05_causal_ablation/04_orthogonal_interventions.py, lines 1 to 45 · score 0.84 · Orthogonal interventions, uniform attention replacement, zeroing FFN, MLP ablation, Head masking, Causal · block oscr:b:1NGJEG5EYXT6C-W · Report a mismatch
  12. [12] § Results › Causal ablation reveals distributed redundancy ↔ src/05_causal_ablation/03_expanded_ablation.py, lines 1 to 47 · score 0.75 · matched random controls, alternative ranking, layer ablation, dose response, Geneformer V2, composite · block oscr:b:57Y7G9FC91TSV-V · Report a mismatch
  13. [13] § Results › Positive control: the pipeline detects planted regulatory signal ↔ src/02_cssi_method/cssi_real_data/20_positive_control_tfs.py, lines 127 to 210 · score 0.75 · HVG evaluable targets, TRRUST direction known, TRRUST entries, best TF, TF AUROC, HVG genes · block oscr:b:0RDG75Q3GSB67-M · Report a mismatch
  14. [14] § Methods › Confound decomposition ↔ src/13_synthetic_validation/20_positive_control_pipeline.py, lines 370 to 417 · score 0.70 · logistic regression, GroupKFold, cross validation, target gene, fitted, splits · block oscr:b:NR1Q9B8H9YGNQ-R · Report a mismatch
  15. [15] § Results › No incremental pairwise value on the perturbation-target prediction task ↔ src/13_synthetic_validation/20_positive_control_pipeline.py, lines 419 to 495 · score 0.68 · plus correlation, logistic regression, GroupKFold, correlation edge, dropout, variance · block oscr:b:W1BH8Z9KB3Z7E-3 · Report a mismatch
  16. [16] § Results › Attention-specific scaling failure and CSSI remedy ↔ src/01_scaling_failure/10_paper_bundle.py, lines 103 to 143 · score 0.67 · attention derived GRN, model tiers, archived, medium, degrades, degradation · block oscr:b:CK2Y3D6NSHXDA-X · Report a mismatch
  17. [17] § Results › Cross-cell-type generalisation of confound pattern ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1140 to 1207 · score 0.64 · full K562 confound, evaluable TFs, TRRUST direct target, battery, GBDT, RPE1 · block oscr:b:Q0CY4H8BPY389-X · Report a mismatch
  18. [18] § Results › Cross-cell-type generalisation of confound pattern ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1140 to 1207 · score 0.64 · full K562 confound, evaluable TFs, TRRUST direct target, battery, GBDT, RPE1 · block oscr:b:G8D5RMZA73BP5-6 · Report a mismatch
  19. [19] § Methods › Scaling and CSSI ↔ src/15_biological/04_cssi_biology.py, lines 1 to 30 · score 0.63 · stratum edge scores, Spearman correlation, Cell State, strata, CSSI, Stratified · block oscr:b:5GTBCFS3Y840C-6 · Report a mismatch
  20. [20] § Methods › Confound decomposition ↔ src/04_confound_decomposition/05_residualization_v2.py, lines 49 to 99 · score 0.63 · Cross fitted residualisation, residual AUROC, OLS, GBDT, regression, splits · block oscr:b:AB83GFZGR2NP0-5 · Report a mismatch
  21. [21] § Methods › Attention extraction and edge score computation ↔ src/05_causal_ablation/04_orthogonal_interventions.py, lines 586 to 651 · score 0.62 · BertSelfAttention, forward hooks, softmax, PyTorch, module, Batch · block oscr:b:PYQB9AFMCHQD5-6 · Report a mismatch
  22. [22] § Results › Evaluation framework: two objectives, five test families ↔ src/13_synthetic_validation/20_positive_control_pipeline.py, lines 1 to 55 · score 0.62 · planted pairwise signal, Trivial baseline, pipeline, synthetic, diagnostics, edge scores · block oscr:b:G1HS3G2FCSK46-B · Report a mismatch
  23. [23] § Methods › Scaling and CSSI ↔ src/15_biological/09_harden_cssi_biology.py, lines 1 to 31 · score 0.62 · CSSI max, Cell State, Leiden, resolution, stratum, cluster · block oscr:b:K0QDF39A4RKPM-Q · Report a mismatch
  24. [24] § Methods › Models and data ↔ src/14_multi_model/07_investigate_models.py, lines 42 to 120 · score 0.61 · scFoundation, single cell foundation, routinely, UCE, BERT, transformers · block oscr:b:Z77JCEHJ0PN2E-D · Report a mismatch
  25. [25] § Methods › Statistical framework ↔ src/15_biological/07_fix_expression.py, lines 138 to 200 · score 0.60 · Bootstrap confidence intervals, master regulator, TF characterization, biological, family, AUROC · block oscr:b:N1PKQRM0NH8QM-L · Report a mismatch
  26. [26] § Methods › Attention extraction and edge score computation ↔ src/03_perturbation_validation/02_attention_first.py, lines 265 to 317 · score 0.60 · output_attentions, evaluation layer, Geneformer V2, PyTorch, hooks, heads · block oscr:b:P6DWCPHVT7BPQ-G · Report a mismatch
  27. [27] § Methods › Perturbation-first validation ↔ src/02_cssi_method/crispri_validation/scripts/01_pipeline.py, lines 436 to 508 · score 0.58 · Mann Whitney, perturbation genes, control cell, threshold, K562, validation · block oscr:b:1QPNSKE9TE83D-Z · Report a mismatch
  28. [28] § Methods › Perturbation-first validation ↔ src/02_cssi_method/10_perturbation_validation.py, lines 81 to 200 · score 0.58 · Mann Whitney, perturbation genes, control cell, LFC, validation, edge · block oscr:b:WY272653DB3AE-9 · Report a mismatch
  29. [29] § Results › Cross-cell-type generalisation of confound pattern ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1326 to 1369 · score 0.58 · RPE1 confound battery, propensity matching, Trivial baselines, Residualisation, incremental, Cross · block oscr:b:EP8ZGN91KEWCJ-4 · Report a mismatch
  30. [30] § Results › Cross-cell-type generalisation of confound pattern ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1326 to 1369 · score 0.58 · RPE1 confound battery, propensity matching, Trivial baselines, Residualisation, incremental, Cross · block oscr:b:S9NA63KC4SS52-1 · Report a mismatch
  31. [31] § Results › Perturbation-first validation reveals gene-level dominance ↔ src/02_cssi_method/crispri_validation/scripts/03_cssi_pipeline.py, lines 872 to 925 · score 0.57 · regulatory information, achieves AUROC, regulatory signal, detectable, interactions, validation · block oscr:b:B9SYKESJ6B8WP-A · Report a mismatch
  32. [32] § Results › Perturbation-first validation reveals gene-level dominance ↔ src/16_statistical_framework/02_multiplicity_sensitivity.py, lines 192 to 259 · score 0.56 · layer perturbation, cross gene, nested, Wilcoxon, L15, CRISPRa · block oscr:b:V06T86F5V5QKE-U · Report a mismatch
  33. [33] § Results › Perturbation-first validation reveals gene-level dominance ↔ src/02_cssi_method/crispri_validation/scripts/03_cssi_pipeline.py, lines 872 to 925 · score 0.56 · layers achieve AUROC, best layer, CRISPRi, interactions, L13, validation · block oscr:b:B9SYKESJ6B8WP-A · Report a mismatch
  34. [34] § Methods › Models and data ↔ src/15_biological/05_context_dependence.py, lines 47 to 122 · score 0.56 · iPSC, Tian, Shifrut, Adamson, neurons, Replogle · block oscr:b:X1E1H36KHH7RX-5 · Report a mismatch
  35. [35] § Results › Attention-specific scaling failure and CSSI remedy ↔ src/02_cssi_method/cssi_real_data/20_positive_control_tfs.py, lines 1 to 52 · score 0.55 · late Geneformer layers, motivated, exploratory, NN, TFs, CSSI · block oscr:b:5J0TDXMG9WYN6-O · Report a mismatch
  36. [36] § Results › No incremental pairwise value on the perturbation-target prediction task ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1277 to 1324 · score 0.55 · Cross fitted residualisation, propensity matched, OLS, covariate, GBDT, raw · block oscr:b:YVEYP8JKCY38A-5 · Report a mismatch
  37. [37] § Results › No incremental pairwise value on the perturbation-target prediction task ↔ src/06_cross_context/09_rpe1_confound_battery.py, lines 1277 to 1324 · score 0.55 · Cross fitted residualisation, propensity matched, OLS, covariate, GBDT, raw · block oscr:b:NPNSGXZ4RMK2E-9 · Report a mismatch
  38. [38] § Methods › Models and data ↔ src/14_multi_model/06_v1_analysis_summary.py, lines 294 to 387 · score 0.54 · single cell foundation, scGPT, cited, architectures, transformers, rank · block oscr:b:ZX3KGNW14S7C6-D · Report a mismatch
  39. [39] § Results › No incremental pairwise value on the perturbation-target prediction task ↔ src/04_confound_decomposition/08_metric_robust_incremental.py, lines 444 to 497 · score 0.52 · joint cross gene, cross perturbation, harder, AUPRC, GBDT, splits · block oscr:b:W9V4X091V4RZJ-V · Report a mismatch
  40. [40] § Results › No incremental pairwise value on the perturbation-target prediction task ↔ src/04_confound_decomposition/08_metric_robust_incremental.py, lines 444 to 497 · score 0.52 · joint cross gene, cross perturbation, harder, AUPRC, GBDT, splits · block oscr:b:X4W8GGRXVMAGQ-R · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 605 lines · 23 KB · no license · 4 matches

  1. #!/usr/bin/env python3
  2. """
  3. Positive control for the perturbation-first evaluation pipeline.
  4. Motivation (reviewer R1.3): the paper reports that correlation and attention
  5. edge scores add no incremental value beyond trivial gene-level baselines for
  6. perturbation-target prediction on Replogle K562 CRISPRi. Reviewer 1 flagged
  7. that without a decisive positive control this null is hard to interpret —
  8. it could reflect limitations of the pipeline rather than absence of signal.
  9. This script builds a synthetic single-cell Perturb-seq dataset with planted
  10. ground-truth regulatory edges, applies the full trivial-baseline +
  11. incremental-value pipeline end-to-end, and checks whether the pipeline
  12. recovers the planted pairwise signal. Three stress variants are run:
  13. Variant A (strong): strong direct edges, minimal indirect effects.
  14. Pipeline should recover edges strongly. Establishes ceiling.
  15. Variant B (weak + indirect): weak direct edges, strong indirect
  16. propagation. Tests whether indirect effects mask pairwise signal.
  17. Variant C (realistic Replogle-like): gene-level AUROC tuned to match
  18. K562 (~0.88). Diagnoses whether the K562 null could persist even
  19. when ground truth exists at a detectable level.
  20. A shuffled-null control is run within each variant: ground-truth is
  21. randomly permuted and the full pipeline is re-run. The shuffled result
  22. must be near-chance — this confirms pipeline gains come from planted
  23. structure, not bookkeeping artefacts.
  24. Deliverable (for paper): Supplementary Note 21, new figure
  25. fig_positive_control.png, and a summary paragraph in Results referencing
  26. the pipeline sensitivity established here.
  27. Usage:
  28. python 20_positive_control_pipeline.py [--variant A|B|C|all]
  29. Output:
  30. results/positive_control/variant_{A,B,C}.json
  31. results/positive_control/shuffled_null_{A,B,C}.json
  32. results/positive_control/summary.json
  33. """
  34. from __future__ import annotations
  35. import argparse
  36. import json
  37. import os
  38. import sys
  39. from dataclasses import dataclass, field, asdict
  40. from pathlib import Path
  41. import numpy as np
  42. from scipy import stats
  43. from sklearn.linear_model import LogisticRegression
  44. from sklearn.metrics import roc_auc_score
  45. from sklearn.model_selection import GroupKFold
  46. # Line-buffered stdout so long runs emit progress
  47. sys.stdout = os.fdopen(sys.stdout.fileno(), "w", 1)
  48. REPO_ROOT = Path(__file__).resolve().parents[2]
  49. RESULTS_DIR = REPO_ROOT / "results" / "positive_control"
  50. RESULTS_DIR.mkdir(parents=True, exist_ok=True)
  51. # ---------------------------------------------------------------------------
  52. # Synthetic GRN generator
  53. # ---------------------------------------------------------------------------
  54. @dataclass
  55. class SyntheticGRNConfig:
  56. n_genes: int = 300
  57. n_tfs: int = 20
  58. direct_targets_per_tf: int = 6 # direct TF -> target edges
  59. indirect_targets_per_tf: int = 8 # TF -> target (via 1 regulator hop)
  60. n_ctrl_cells: int = 5000
  61. n_pert_cells_per_condition: int = 150
  62. n_perturbations: int = 20 # knockdowns; each TF is perturbed once
  63. edge_strength: float = 1.5 # direct edge coefficient magnitude
  64. indirect_strength: float = 0.8 # indirect propagation coefficient
  65. noise_scale: float = 0.8
  66. dropout_p: float = 0.1
  67. # If True, z-score each gene post-hoc so variance is uninformative.
  68. # Used in positive-control variants to isolate pairwise signal.
  69. z_score_genes: bool = False
  70. # If True, restrict the positive class to direct targets only (rather
  71. # than all DE hits). This tests whether correlation edges predict
  72. # *direct* regulation vs. indirect propagation.
  73. direct_only_positives: bool = False
  74. # Variant tag for reproducibility of the stress suite.
  75. variant: str = "A"
  76. seed: int = 2026
  77. def build_grn(cfg: SyntheticGRNConfig, rng: np.random.Generator):
  78. """Build a sparse hierarchical GRN.
  79. Returns:
  80. tf_idx: indices of TFs (shape [n_tfs])
  81. direct: dict TF_idx -> list of direct target gene indices
  82. indirect: dict TF_idx -> list of indirect target gene indices
  83. weight_mat: (n_genes, n_genes) direct TF->target weights
  84. indirect_weight_mat: (n_genes, n_genes) indirect effect weights
  85. """
  86. n = cfg.n_genes
  87. gene_pool = np.arange(n)
  88. tf_idx = rng.choice(gene_pool, size=cfg.n_tfs, replace=False)
  89. non_tf = np.setdiff1d(gene_pool, tf_idx)
  90. direct = {}
  91. indirect = {}
  92. weight_mat = np.zeros((n, n), dtype=np.float32)
  93. indirect_weight_mat = np.zeros((n, n), dtype=np.float32)
  94. for tf in tf_idx:
  95. # Direct targets — TFs regulate non-TF genes
  96. dtargets = rng.choice(
  97. non_tf, size=cfg.direct_targets_per_tf, replace=False
  98. )
  99. # Mix of activation / repression
  100. signs = rng.choice([-1.0, 1.0], size=len(dtargets))
  101. magnitudes = rng.uniform(0.7, 1.3, size=len(dtargets)) * cfg.edge_strength
  102. for tgt, s, m in zip(dtargets, signs, magnitudes):
  103. weight_mat[tf, tgt] = s * m
  104. direct[int(tf)] = dtargets.tolist()
  105. # Indirect targets: not in direct set, reached via a single-hop
  106. # regulator. These contribute to DE on perturbation but are not
  107. # direct regulatory edges.
  108. candidate_pool = np.setdiff1d(non_tf, dtargets)
  109. itargets = rng.choice(
  110. candidate_pool, size=cfg.indirect_targets_per_tf, replace=False
  111. )
  112. signs_i = rng.choice([-1.0, 1.0], size=len(itargets))
  113. mags_i = rng.uniform(0.6, 1.2, size=len(itargets)) * cfg.indirect_strength
  114. for tgt, s, m in zip(itargets, signs_i, mags_i):
  115. indirect_weight_mat[tf, tgt] = s * m
  116. indirect[int(tf)] = itargets.tolist()
  117. return tf_idx, direct, indirect, weight_mat, indirect_weight_mat
  118. def simulate_control_cells(
  119. cfg: SyntheticGRNConfig,
  120. rng: np.random.Generator,
  121. weight_mat: np.ndarray,
  122. indirect_weight_mat: np.ndarray,
  123. ) -> np.ndarray:
  124. """Simulate baseline control expression.
  125. Model: baseline TF activity is drawn from a heavy-tailed distribution,
  126. target expression is a linear combination of regulator activities plus
  127. Gaussian noise plus dropout. TF expression equals TF activity directly.
  128. We optionally z-score each gene post-hoc to control whether gene-level
  129. variance carries information about direct-target status. This is
  130. essential for the positive-control design: variant A z-scores to make
  131. variance uninformative, so any incremental value from correlation
  132. edges reflects genuine pairwise signal detection.
  133. """
  134. n = cfg.n_genes
  135. n_cells = cfg.n_ctrl_cells
  136. # Heavy-tailed baseline TF activities for each gene slot
  137. tf_activity = rng.standard_t(df=3, size=(n_cells, n)).astype(np.float32) * 0.8
  138. # Target expression comes from regulator activity
  139. expr = tf_activity @ (weight_mat + indirect_weight_mat)
  140. # TFs' own expression is their activity (direct readout)
  141. tf_mask = weight_mat.sum(axis=1) != 0 # rows with outgoing edges are TFs
  142. expr[:, tf_mask] = tf_activity[:, tf_mask]
  143. # Add small random mean shift (identifies the gene but does not correlate
  144. # with direct-target status)
  145. gene_means = rng.normal(0, 0.3, size=n).astype(np.float32)
  146. expr = expr + gene_means
  147. # Gaussian noise (additive)
  148. expr = expr + rng.normal(
  149. 0, cfg.noise_scale, size=expr.shape
  150. ).astype(np.float32)
  151. # Dropout
  152. mask = rng.uniform(size=expr.shape) < cfg.dropout_p
  153. expr[mask] = 0.0
  154. # Optionally z-score per gene so variance is uninformative.
  155. if getattr(cfg, "z_score_genes", False):
  156. std = expr.std(axis=0)
  157. std[std == 0] = 1.0
  158. expr = (expr - expr.mean(axis=0)) / std
  159. return expr.astype(np.float32)
  160. def simulate_perturbation(
  161. cfg: SyntheticGRNConfig,
  162. rng: np.random.Generator,
  163. weight_mat: np.ndarray,
  164. indirect_weight_mat: np.ndarray,
  165. pert_tf: int,
  166. ) -> np.ndarray:
  167. """Simulate cells where the given TF is knocked down.
  168. Knockdown: set pert_tf activity to near zero and re-simulate expression.
  169. """
  170. n = cfg.n_genes
  171. n_cells = cfg.n_pert_cells_per_condition
  172. tf_activity = rng.standard_t(df=3, size=(n_cells, n)) * 0.5
  173. # Knockdown: zero out the perturbed TF's activity
  174. tf_activity[:, pert_tf] *= 0.05
  175. expr = tf_activity @ (weight_mat + indirect_weight_mat)
  176. gene_means = 0.0 # means are cell-invariant, already in baseline
  177. expr = expr + gene_means
  178. expr = expr + rng.normal(0, cfg.noise_scale, size=expr.shape).astype(np.float32)
  179. mask = rng.uniform(size=expr.shape) < cfg.dropout_p
  180. expr[mask] = 0.0
  181. return expr.astype(np.float32)
  182. # ---------------------------------------------------------------------------
  183. # Pipeline components: DE calling, gene-level features, correlation edges
  184. # ---------------------------------------------------------------------------
  185. def compute_gene_features(ctrl_expr: np.ndarray) -> np.ndarray:
  186. """Compute trivial gene-level features from control cells.
  187. Returns array (n_genes, 3): variance, mean, dropout rate.
  188. """
  189. var = ctrl_expr.var(axis=0)
  190. mean = ctrl_expr.mean(axis=0)
  191. dropout = (ctrl_expr == 0).mean(axis=0)
  192. return np.stack([var, mean, dropout], axis=1)
  193. def compute_correlation_edges(ctrl_expr: np.ndarray) -> np.ndarray:
  194. """Compute gene-by-gene Spearman correlation edge scores on control cells.
  195. Uses fast rank-based Pearson as an approximation of Spearman.
  196. Returns (n_genes, n_genes) symmetric matrix.
  197. """
  198. # Rank each gene independently
  199. ranks = stats.rankdata(ctrl_expr, axis=0)
  200. ranks = ranks - ranks.mean(axis=0)
  201. std = ranks.std(axis=0)
  202. std[std == 0] = 1.0
  203. ranks = ranks / std
  204. corr = (ranks.T @ ranks) / ranks.shape[0]
  205. # Use absolute correlation as edge score (magnitude of association)
  206. return np.abs(corr).astype(np.float32)
  207. def call_de(
  208. ctrl_expr: np.ndarray,
  209. pert_expr: np.ndarray,
  210. lfc_threshold: float = 0.5,
  211. alpha: float = 0.05,
  212. ) -> np.ndarray:
  213. """Return boolean array (n_genes,) of DE-hit targets.
  214. Uses Welch's t-test + LFC threshold (crude, but matches paper pipeline).
  215. """
  216. n_genes = ctrl_expr.shape[1]
  217. de_hits = np.zeros(n_genes, dtype=bool)
  218. # Simple LFC in log1p-space
  219. mean_c = np.log1p(np.abs(ctrl_expr).mean(axis=0) + 1e-3)
  220. mean_p = np.log1p(np.abs(pert_expr).mean(axis=0) + 1e-3)
  221. lfc = mean_p - mean_c
  222. # Welch's t across genes
  223. t, p = stats.ttest_ind(pert_expr, ctrl_expr, axis=0, equal_var=False)
  224. de_hits = (np.abs(lfc) > lfc_threshold) & (p < alpha)
  225. return de_hits
  226. # ---------------------------------------------------------------------------
  227. # Main experiment
  228. # ---------------------------------------------------------------------------
  229. @dataclass
  230. class VariantResult:
  231. variant: str
  232. config: dict
  233. n_perturbations: int = 0
  234. n_evaluable: int = 0
  235. gene_only_auroc: float = float("nan")
  236. gene_plus_corr_auroc: float = float("nan")
  237. delta_auroc_corr: float = float("nan")
  238. delta_auroc_ci: tuple = field(default_factory=lambda: (float("nan"), float("nan")))
  239. per_tf_corr_auroc: dict = field(default_factory=dict)
  240. variance_baseline_auroc: float = float("nan")
  241. mean_baseline_auroc: float = float("nan")
  242. dropout_baseline_auroc: float = float("nan")
  243. positive_rate: float = float("nan")
  244. notes: str = ""
  245. def run_variant(cfg: SyntheticGRNConfig, shuffle_ground_truth: bool = False) -> VariantResult:
  246. """Run one stress variant and return summary metrics.
  247. If shuffle_ground_truth, the TF->target mapping is randomly permuted
  248. after simulation so that simulated DE still occurs but correlation
  249. edges no longer map to ``true'' direct targets. This provides the
  250. negative control: pipeline should produce zero incremental value.
  251. """
  252. rng = np.random.default_rng(cfg.seed + (1_000_000 if shuffle_ground_truth else 0))
  253. tf_idx, direct, indirect, weight_mat, indirect_weight_mat = build_grn(cfg, rng)
  254. print(f"[{cfg.variant}] Built GRN: {cfg.n_genes} genes, {cfg.n_tfs} TFs, "
  255. f"{sum(len(v) for v in direct.values())} direct edges")
  256. # Simulate control cells
  257. ctrl_expr = simulate_control_cells(cfg, rng, weight_mat, indirect_weight_mat)
  258. print(f"[{cfg.variant}] Simulated {ctrl_expr.shape[0]} control cells")
  259. # Gene-level features + correlation edges from control cells
  260. gene_feats = compute_gene_features(ctrl_expr)
  261. corr_edges = compute_correlation_edges(ctrl_expr)
  262. # Optionally shuffle ground truth: permute target labels per TF
  263. # so DE positives no longer match the edges that contributed to them.
  264. if shuffle_ground_truth:
  265. all_non_tf = [g for g in range(cfg.n_genes) if g not in set(tf_idx.tolist())]
  266. shuffled_direct = {}
  267. for tf in direct:
  268. shuffled_direct[tf] = list(
  269. rng.choice(all_non_tf, size=len(direct[tf]), replace=False)
  270. )
  271. # Note: we keep the actual simulation edges in place (so DE still
  272. # happens biologically), but we treat shuffled_direct as the
  273. # "ground truth direct set". Any incremental value on this shuffled
  274. # truth is spurious.
  275. ground_truth_direct = shuffled_direct
  276. else:
  277. ground_truth_direct = {int(tf): list(direct[int(tf)]) for tf in direct}
  278. # Simulate each perturbation
  279. pert_results = [] # (tf, de_mask, per_gene_pair_rows)
  280. rows_X = []
  281. rows_y = []
  282. rows_group = []
  283. tfs_to_perturb = tf_idx[: cfg.n_perturbations]
  284. for k, tf in enumerate(tfs_to_perturb):
  285. tf_int = int(tf)
  286. pert_expr = simulate_perturbation(
  287. cfg, rng, weight_mat, indirect_weight_mat, pert_tf=tf_int
  288. )
  289. # Compute DE hits empirically (used for descriptive stats)
  290. de_hits_empirical = call_de(ctrl_expr, pert_expr)
  291. if cfg.direct_only_positives:
  292. # Positive class = direct ground-truth targets only.
  293. # This is the cleanest test of whether the pipeline can
  294. # detect direct pairwise regulation.
  295. positives = np.zeros(cfg.n_genes, dtype=bool)
  296. for tgt in ground_truth_direct.get(tf_int, []):
  297. positives[tgt] = True
  298. label_mask = positives
  299. else:
  300. label_mask = de_hits_empirical
  301. n_pos = int(label_mask.sum())
  302. if n_pos < 3 or n_pos > cfg.n_genes - 3:
  303. continue
  304. pert_results.append((tf_int, label_mask, n_pos))
  305. # Build per-perturbation rows: for each non-self target gene,
  306. # features = [tf_var, tf_mean, tf_dropout, tgt_var, tgt_mean, tgt_dropout, corr_edge]
  307. for tgt in range(cfg.n_genes):
  308. if tgt == tf_int:
  309. continue
  310. X_row = [
  311. gene_feats[tf_int, 0], gene_feats[tf_int, 1], gene_feats[tf_int, 2],
  312. gene_feats[tgt, 0], gene_feats[tgt, 1], gene_feats[tgt, 2],
  313. corr_edges[tf_int, tgt],
  314. ]
  315. rows_X.append(X_row)
  316. rows_y.append(int(label_mask[tgt]))
  317. rows_group.append(tf_int)
  318. X = np.array(rows_X, dtype=np.float32)
  319. y = np.array(rows_y, dtype=np.int32)
  320. groups = np.array(rows_group, dtype=np.int32)
  321. if len(pert_results) < 3 or y.sum() == 0:
  322. return VariantResult(
  323. variant=cfg.variant,
  324. config=asdict(cfg),
  325. notes="Too few evaluable perturbations or no DE hits",
  326. )
  327. positive_rate = float(y.mean())
  328. print(f"[{cfg.variant}] {len(pert_results)} perturbations, "
  329. f"{len(y)} gene-pair rows, positive rate {positive_rate:.3f}")
  330. # 5-fold GroupKFold by source TF
  331. n_splits = min(5, len(set(groups)))
  332. gkf = GroupKFold(n_splits=n_splits)
  333. def eval_model(feat_cols):
  334. """Cross-validated mean AUROC across folds on the given feature cols."""
  335. aurocs = []
  336. for tr, te in gkf.split(X, y, groups):
  337. X_tr = X[tr][:, feat_cols]
  338. X_te = X[te][:, feat_cols]
  339. y_tr = y[tr]
  340. y_te = y[te]
  341. if y_te.sum() == 0 or y_te.sum() == len(y_te):
  342. continue
  343. clf = LogisticRegression(max_iter=1000, solver="liblinear")
  344. clf.fit(X_tr, y_tr)
  345. p = clf.predict_proba(X_te)[:, 1]
  346. aurocs.append(roc_auc_score(y_te, p))
  347. return float(np.mean(aurocs)) if aurocs else float("nan")
  348. # Feature column indices
  349. GENE_COLS = [0, 1, 2, 3, 4, 5]
  350. CORR_COL = [6]
  351. gene_only = eval_model(GENE_COLS)
  352. gene_plus_corr = eval_model(GENE_COLS + CORR_COL)
  353. delta = gene_plus_corr - gene_only
  354. # Bootstrap delta-AUROC CI (100 resamples over perturbations)
  355. rng_boot = np.random.default_rng(cfg.seed + 7777)
  356. deltas = []
  357. unique_tfs = sorted(set(groups.tolist()))
  358. for _ in range(100):
  359. boot_tfs = rng_boot.choice(unique_tfs, size=len(unique_tfs), replace=True)
  360. mask = np.isin(groups, boot_tfs)
  361. if y[mask].sum() == 0:
  362. continue
  363. Xb = X[mask]
  364. yb = y[mask]
  365. gb = groups[mask]
  366. try:
  367. gkf_b = GroupKFold(n_splits=min(3, len(set(gb))))
  368. aur_g, aur_gc = [], []
  369. for tr, te in gkf_b.split(Xb, yb, gb):
  370. if yb[te].sum() == 0 or yb[te].sum() == len(yb[te]):
  371. continue
  372. c1 = LogisticRegression(max_iter=1000, solver="liblinear")
  373. c1.fit(Xb[tr][:, GENE_COLS], yb[tr])
  374. aur_g.append(roc_auc_score(yb[te], c1.predict_proba(Xb[te][:, GENE_COLS])[:, 1]))
  375. c2 = LogisticRegression(max_iter=1000, solver="liblinear")
  376. c2.fit(Xb[tr][:, GENE_COLS + CORR_COL], yb[tr])
  377. aur_gc.append(roc_auc_score(yb[te], c2.predict_proba(Xb[te][:, GENE_COLS + CORR_COL])[:, 1]))
  378. if aur_g and aur_gc:
  379. deltas.append(np.mean(aur_gc) - np.mean(aur_g))
  380. except Exception:
  381. continue
  382. if deltas:
  383. ci_low, ci_high = float(np.percentile(deltas, 2.5)), float(np.percentile(deltas, 97.5))
  384. else:
  385. ci_low, ci_high = float("nan"), float("nan")
  386. # Trivial baselines: each gene-level feature alone
  387. def single_feature_auroc(col):
  388. scores = X[:, col]
  389. return float(roc_auc_score(y, scores))
  390. variance_auroc = single_feature_auroc(3) # target variance
  391. mean_auroc = single_feature_auroc(4)
  392. dropout_auroc = 1.0 - single_feature_auroc(5) # low dropout = more detectable
  393. # Per-TF correlation-edge AUROC (marginal, no gene features)
  394. per_tf_corr_auroc = {}
  395. for tf_int_inner, label_mask_inner, _ in pert_results:
  396. # Use corr_edges[tf, :] as scores for each target
  397. scores = corr_edges[tf_int_inner]
  398. try:
  399. aur = roc_auc_score(label_mask_inner.astype(int), scores)
  400. except ValueError:
  401. aur = float("nan")
  402. per_tf_corr_auroc[int(tf_int_inner)] = float(aur)
  403. res = VariantResult(
  404. variant=cfg.variant,
  405. config=asdict(cfg),
  406. n_perturbations=cfg.n_perturbations,
  407. n_evaluable=len(pert_results),
  408. gene_only_auroc=gene_only,
  409. gene_plus_corr_auroc=gene_plus_corr,
  410. delta_auroc_corr=delta,
  411. delta_auroc_ci=(ci_low, ci_high),
  412. per_tf_corr_auroc=per_tf_corr_auroc,
  413. variance_baseline_auroc=variance_auroc,
  414. mean_baseline_auroc=mean_auroc,
  415. dropout_baseline_auroc=dropout_auroc,
  416. positive_rate=positive_rate,
  417. notes="shuffled ground truth" if shuffle_ground_truth else "planted ground truth",
  418. )
  419. return res
  420. def main():
  421. parser = argparse.ArgumentParser()
  422. parser.add_argument("--variant", default="all", choices=["A", "B", "C", "all"])
  423. args = parser.parse_args()
  424. # Three stress variants.
  425. # All three z-score gene expression so variance is uninformative —
  426. # this is the positive-control design: any delta_AUROC > 0 must come
  427. # from pairwise (correlation) information, not univariate baselines.
  428. # The variants then stress different aspects:
  429. # A: strong, clean direct regulation with minimal indirect.
  430. # Pipeline ceiling test.
  431. # B: weak direct edges + strong indirect propagation.
  432. # Tests whether indirect effects mask pairwise signal.
  433. # C: realistic balance of direct + indirect + noise.
  434. # Closest to real K562-like regime.
  435. variants = {
  436. "A": SyntheticGRNConfig(
  437. variant="A",
  438. edge_strength=2.2,
  439. indirect_strength=0.1,
  440. noise_scale=0.5,
  441. direct_targets_per_tf=8,
  442. indirect_targets_per_tf=2,
  443. z_score_genes=True,
  444. direct_only_positives=True, # positive class = direct targets only
  445. seed=2026,
  446. ),
  447. "B": SyntheticGRNConfig(
  448. variant="B",
  449. edge_strength=0.8,
  450. indirect_strength=1.6,
  451. noise_scale=0.9,
  452. direct_targets_per_tf=5,
  453. indirect_targets_per_tf=15,
  454. z_score_genes=True,
  455. direct_only_positives=True,
  456. seed=2027,
  457. ),
  458. "C": SyntheticGRNConfig(
  459. variant="C",
  460. edge_strength=1.2,
  461. indirect_strength=0.8,
  462. noise_scale=0.8,
  463. direct_targets_per_tf=6,
  464. indirect_targets_per_tf=8,
  465. z_score_genes=True,
  466. direct_only_positives=True,
  467. seed=2028,
  468. ),
  469. }
  470. if args.variant != "all":
  471. variants = {args.variant: variants[args.variant]}
  472. summary = {}
  473. for vname, cfg in variants.items():
  474. print(f"\n===== Variant {vname}: {cfg.notes if hasattr(cfg, 'notes') else ''} =====")
  475. # Planted ground-truth run
  476. res = run_variant(cfg, shuffle_ground_truth=False)
  477. out_path = RESULTS_DIR / f"variant_{vname}.json"
  478. with open(out_path, "w") as f:
  479. # Convert tuple to list for JSON
  480. d = asdict(res)
  481. d["delta_auroc_ci"] = list(d["delta_auroc_ci"])
  482. json.dump(d, f, indent=2, default=float)
  483. print(f" gene_only AUROC: {res.gene_only_auroc:.4f}")
  484. print(f" gene+corr AUROC: {res.gene_plus_corr_auroc:.4f}")
  485. print(f" delta_auroc: {res.delta_auroc_corr:+.4f} "
  486. f"[{res.delta_auroc_ci[0]:+.4f}, {res.delta_auroc_ci[1]:+.4f}]")
  487. print(f" variance baseline: {res.variance_baseline_auroc:.4f}")
  488. print(f" mean ctrl baseline: {res.mean_baseline_auroc:.4f}")
  489. # Shuffled-null control
  490. print(f"\n --- shuffled null for variant {vname} ---")
  491. res_null = run_variant(cfg, shuffle_ground_truth=True)
  492. null_path = RESULTS_DIR / f"shuffled_null_{vname}.json"
  493. with open(null_path, "w") as f:
  494. d = asdict(res_null)
  495. d["delta_auroc_ci"] = list(d["delta_auroc_ci"])
  496. json.dump(d, f, indent=2, default=float)
  497. print(f" shuffled gene_only: {res_null.gene_only_auroc:.4f}")
  498. print(f" shuffled gene+corr: {res_null.gene_plus_corr_auroc:.4f}")
  499. print(f" shuffled delta: {res_null.delta_auroc_corr:+.4f}")
  500. summary[vname] = {
  501. "planted": {
  502. "gene_only": res.gene_only_auroc,
  503. "gene_plus_corr": res.gene_plus_corr_auroc,
  504. "delta": res.delta_auroc_corr,
  505. "delta_ci": list(res.delta_auroc_ci),
  506. "variance_baseline": res.variance_baseline_auroc,
  507. "n_evaluable": res.n_evaluable,
  508. "positive_rate": res.positive_rate,
  509. },
  510. "shuffled_null": {
  511. "gene_only": res_null.gene_only_auroc,
  512. "gene_plus_corr": res_null.gene_plus_corr_auroc,
  513. "delta": res_null.delta_auroc_corr,
  514. },
  515. }
  516. with open(RESULTS_DIR / "summary.json", "w") as f:
  517. json.dump(summary, f, indent=2, default=float)
  518. print(f"\nDone. Results in {RESULTS_DIR}")
  519. if __name__ == "__main__":
  520. main()

20_positive_control_pipeline.py at commit 40e6da0, no license · at the source

Overview

Authors: Ihor Kendiukhov1
  1. Institute of Medical Genetics and Applied Genomics, University of Tübingen,Tübingen, Germany
Institutions: University of Tübingen (Germany)
Journal: BMC genomics, volume 27, issue 1, article 634
Dates: received 10 March 2026; accepted 14 May 2026; published online 22 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1186/s12864-026-12965-8 · PMID 42482180 · PMCID PMC13390336 · OpenAlex W7169877416
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: human (organism), cellular / molecular (subfield)
Methods: Connectivity, Statistics, Machine learning
Keywords: Single-cell foundation models, Mechanistic interpretability, Gene regulatory networks, Attention mechanisms, ScGPT, Geneformer, Perturbation prediction, CRISPR screens, Benchmarking
MeSH: Computational Biology*, Gene Regulatory Networks*, Single-Cell Analysis*, Humans (* major topic)
Topic: Single-cell and spatial transcriptomics (Molecular Biology, Biochemistry, Genetics and Molecular Biology), according to OpenAlex
Funding: Eberhard Karls Universität Tübingen (1020)
Citations: not cited yet (Europe PMC); 32 references in the paper

Abstract

Background: Single-cell foundation models such as scGPT and Geneformer are increasingly used for gene regulatory network (GRN) inference, with attention-derived edge scores routinely interpreted as regulatory proxies. Prior benchmarks have evaluated curated-reference recovery but have not systematically tested whether attention adds information beyond expression statistics for predicting the outcomes of genetic perturbations, nor whether attention-identified “regulatory” components are causally required for such predictions. This gap matters because the NLP interpretability literature has established that attention weights do not reliably indicate feature importance, and biological foundation models are being deployed without analogous scrutiny.

Methods: We present an evaluation framework comprising thirty-seven analyses and 153 statistical tests under Benjamini-Hochberg FDR correction, spanning two architectures (scGPT, Geneformer V2-316M), four cell types (K562, RPE1, primary T cells, iPSC neurons), and two perturbation modalities (CRISPRi, CRISPRa). The framework separates two objectives: (A) mechanistic interpretability / GRN recovery against curated references, and (B) perturbation-target prediction, i.e. classifying which genes show differential expression after a CRISPR perturbation. Five test families—trivial-baseline comparison, conditional incremental-value testing, residualisation and propensity matching, causal ablation with intervention-fidelity diagnostics, and cross-context replication—address Objective B, supplemented by a synthetic positive control establishing pipeline sensitivity.

Results: Attention patterns encode layer-specific biological structure—protein–protein interactions in early layers, transcriptional regulation in late layers—and Cell-State Stratified Interpretability (CSSI) exploits this structure to improve curated GRN recovery up to on the Objective A task. On Objective B, however, attention-derived edge scores add no incremental value beyond trivial gene-level features (variance, mean expression, dropout rate): gene-level baselines outperform both attention and correlation edges (AUROC 0.81–0.88 versus 0.70), augmenting gene-level predictors with pairwise edges produces AUROC of to across 559,720 perturbation–gene observations, and causal ablation of TRRUST-ranked attention heads produces no degradation across three independent intervention channels. The attention–correlation relationship is context-dependent (equal in K562 CRISPRi, worse in CRISPRa, better in RPE1), but gene-level dominance on Objective B is universal across both cell types where adequate power is available.

Conclusions: Attention patterns in single-cell foundation models encode biologically structured information, including layer-specific regulatory signals recoverable via CSSI, but provide no unique predictive information beyond simple gene-level statistics for the perturbation-target prediction task. The paper thus offers both a cautionary finding for Objective B and a constructive method (CSSI) for Objective A. Practitioners should apply trivial-baseline and incremental-value tests before claiming pairwise regulatory signal, and should stratify by cell state when extracting attention-derived GRNs.

Supplementary Information: The online version contains supplementary material available at 10.1186/s12864-026-12965-8.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 40 matches between paragraphs and lines of code.

Zenodo 18701417

License: CC-BY-4.0
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Size: 1 file
Software Heritage: not checked
Found in: “Data availability”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (164 files), pandas (160 files), SciPy (108 files), Matplotlib (98 files), scikit-learn (93 files), Scanpy (85 files), PyTorch (40 files), Hugging Face Transformers (37 files), statsmodels (34 files), seaborn (27 files), h5py (25 files), anndata (18 files), NetworkX (4 files), igraph (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
  • 27 September 2026: the link answers (HTTP 200)
188 files

Biodyn-AI/biomechinterp-framework

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 40e6da0ae21a768dbbcb2bfa62539845f0ef7c88, 14 April 2026
Languages: Python (193), Shell (1)
Size: 541 files, 194 scripts
Software Heritage: not archived
Found in: “Data availability”
Holds: README, environment (src/environment.yml, src/requirements.txt), tests
Not found: license file, CITATION.cff, continuous integration, documentation
Tools: NumPy (169 files), pandas (163 files), SciPy (109 files), Matplotlib (100 files), scikit-learn (97 files), Scanpy (85 files), PyTorch (41 files), Hugging Face Transformers (37 files), statsmodels (34 files), h5py (27 files), seaborn (27 files), anndata (18 files), NetworkX (4 files), igraph (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
195 files

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 381 scripts, each with its path and the digest of its content;
  • 40 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability

All analysis scripts, data processing pipelines, source data for all figures and tables, and reproducibility instructions are deposited at Zenodo (https://doi.org/10.5281/zenodo.18701417). Complete analysis code, figure generation scripts, CSSI implementation, and computational environment specifications are available at https://github.com/Biodyn-AI/biomechinterp-framework and archived at Zenodo. All primary datasets (Tabula Sapiens, Replogle Perturb-seq, Dixit/Adamson/Shifrut datasets) are publicly available from their original sources.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 1 author, 9 keywords, 4 MeSH terms, 1 funder, 19 references.

Cite

This paper

Kendiukhov, I. (2026). Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction. BMC genomics, 27(1), 634. https://doi.org/10.1186/s12864-026-12965-8

BibTeX

@article{kendiukhov2026systematic,
author = {Kendiukhov, Ihor},
title = {{Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction}},
journal = {BMC genomics},
year = {2026},
month = jul,
volume = {27},
number = {1},
pages = {634},
publisher = {BMC},
issn = {1471-2164},
doi = {10.1186/s12864-026-12965-8},
url = {https://doi.org/10.1186/s12864-026-12965-8},
pmid = {42482180},
pmcid = {PMC13390336}
}

RIS

TY - JOUR
AU - Kendiukhov, Ihor
TI - Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction
T2 - BMC genomics
J2 - BMC Genomics
PY - 2026
DA - 2026/07/22
VL - 27
IS - 1
SP - 634
SN - 1471-2164
PB - BMC
DO - 10.1186/s12864-026-12965-8
UR - https://doi.org/10.1186/s12864-026-12965-8
LA - en
ER -

CSL-JSON

{
"id": "10.1186/s12864-026-12965-8",
"type": "article-journal",
"title": "Systematic evaluation of single-cell foundation model interpretability: attention-derived edge scores add no incremental value over gene-level features for perturbation-target prediction",
"container-title": "BMC genomics",
"author": [
{
"family": "Kendiukhov",
"given": "Ihor"
}
],
"container-title-short": "BMC Genomics",
"volume": "27",
"issue": "1",
"page": "634",
"DOI": "10.1186/s12864-026-12965-8",
"PMID": "42482180",
"PMCID": "PMC13390336",
"ISSN": "1471-2164",
"publisher": "BMC",
"URL": "https://doi.org/10.1186/s12864-026-12965-8",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
22
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1093/nar/gkag706 [code]
scDifformer: diffusion-based post-training for virtual cell modeling across large-scale single-cell data.
Journal: Nucleic acids research
In common: anndata, Scanpy, NetworkX, 9 other tools, 5 references
[2] doi:10.1016/j.xgen.2024.100739 [code]
CSI-GEP: A GPU-based unsupervised machine learning approach for recovering gene expression programs in atlas-scale single-cell RNA-seq data.
Journal: Cell genomics
In common: anndata, Scanpy, igraph, 10 other tools, cellular / molecular, 3 references
[3] doi:10.64898/2026.03.30.714220 [code]
An integrated single cell and spatial omics atlas of human prenatal development
Journal: bioRxiv (preprint)
In common: Hugging Face Transformers, anndata, Scanpy, 10 other tools, 2 references
[4] doi:10.1038/s41592-026-03034-9 [code]
3d-OT: a deep geometry-aware framework for heterogeneous slices alignment of spatial multi-omics.
Journal: Nature methods
In common: anndata, Scanpy, NetworkX, 9 other tools, 3 references
[5] doi:10.1093/bib/bbag601 [code]
ADInteractome: discovering Alzheimer's disease-associated gene interactions from single-nucleus RNA-seq data through neural interaction states.
Journal: Briefings in bioinformatics
In common: Hugging Face Transformers, anndata, Scanpy, 8 other tools, cellular / molecular, 2 references
[6] doi:10.1101/gr.280803.125 [code]
Unified integration of spatial transcriptomics across platforms with LLOKI.
Journal: Genome research
In common: anndata, Scanpy, PyTorch, 4 other tools, 5 references
[7] doi:10.1093/bib/bbag259 [code]
PVAED: prior-guided variational autoencoders with diffusion denoising for interpretable single-cell representation learning.
Journal: Briefings in bioinformatics
In common: anndata, Scanpy, NetworkX, 6 other tools, 4 references
[8] doi:10.1093/nar/gkaf1473 [code]
STAN, a computational framework for inferring spatially informed transcription factor activity.
Journal: Nucleic acids research
In common: Hugging Face Transformers, anndata, Scanpy, 9 other tools, cellular / molecular, 1 reference
[9] doi:10.1016/j.xgen.2026.101217 [code]
ProtoCloud: A prototypical self-explaining model for single-cell analysis.
Journal: Cell genomics
In common: anndata, Scanpy, h5py, 7 other tools, cellular / molecular, 3 references
[10] doi:10.1038/s41467-026-71759-4 [code]
CellNiche represents cellular microenvironments in atlas-scale spatial omics data with contrastive learning.
Journal: Nature communications
In common: anndata, Scanpy, NetworkX, 7 other tools, cellular / molecular, 3 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8.svg)](https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8.svg
   :target: https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8"><img src="https://openscicode.org/badge/doi/10.1186/s12864-026-12965-8.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.