OSCR

A multimodal approach for visualizing and identifying electrophysiological cell types in vivo.

Code ↔ Paper

8 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 8 matches
  1. [1] § Methods › Weighted nearest neighbor (WNN) algorithm › β-variational autoencoder and multi-layer perceptron identification architectures ↔ PairOfVAEs_cuda.ipynb, lines 1020–1117 · score 0.96 · Adam optimizer, MLPClassifier, ReLU, hidden layer, latent dim, latent embedding
  2. [2] § Methods › Weighted nearest neighbor (WNN) algorithm › Identification comparison ↔ PairOfVAEs_cuda.ipynb, lines 1119–1226 · score 0.64 · cross validation, confusion matrices, iteration, split, transformation, VAE
  3. [3] § Methods › Weighted nearest neighbor (WNN) algorithm › Distributional shift in waveform shape ↔ juxtacellular/processJianing.ipynb, lines 795–870 · score 0.62 · max_depth, n_estimators, classifier accuracy, trained, cell
  4. [4] § Results › Interpretable representations from PhysMAP enable biological insight ↔ juxtacellular/processJianing.ipynb, lines 127–165 · score 0.61 · inter spike interval, peri stimulus, histograms, latency, onset, mouse
  5. [5] § Methods › Statistics & reproducibility › Extracellular Mouse A1 Dataset ↔ InvivoA1/processSantiago.r, lines 1–71 · score 0.56 · spike quality, ISI violation, amplitude, waveform, cells
  6. [6] § Methods › Weighted nearest neighbor (WNN) algorithm › Identification algorithm ↔ juxtacellular/processJianing.ipynb, lines 795–870 · score 0.55 · GBM classifier, balanced accuracy, split, classes, train, embedded
  7. [7] § Methods › Statistics & reproducibility › Visual Behavior dataset (Allen Institute) ↔ InvivoA1/processSantiago.r, lines 1–71 · score 0.52 · ISI violation, spike waveform, amplitude, quality
  8. [8] § Methods › Weighted nearest neighbor (WNN) algorithm › Joint embedding and label transfer of Ultras and Visual Behavior ↔ juxtacellular/processJianing.r, lines 152–199 · score 0.50 · log ratio, UMAP embedding, identities, PSTH, waveform, cell

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 1,152 lines · 37 KB · no license · 3 matches

  1. # %% [markdown]
  2. # # PhysMAP: Process Jianing Data
  3. #
  4. # This notebook ports the R analysis from `processJianing.r` to Python using Scanpy and Muon.
  5. #
  6. # **Original R pipeline:**
  7. # - Seurat v4 for single-cell analysis
  8. # - Multi-modal integration via FindMultiModalNeighbors (WNN)
  9. #
  10. # **Python implementation:**
  11. # - Scanpy for single-modality processing
  12. # - Muon for multi-modal data handling and integration
  13. # %% [markdown]
  14. # ## 1. Imports and Configuration
  15. # %%
  16. import scanpy as sc
  17. import muon as mu
  18. import numpy as np
  19. import pandas as pd
  20. from scipy.io import loadmat
  21. import matplotlib.pyplot as plt
  22. import seaborn as sns
  23. from anndata import AnnData
  24. from muon import MuData
  25. from sklearn.preprocessing import StandardScaler
  26. from sklearn.decomposition import PCA
  27. import warnings
  28. warnings.filterwarnings('ignore')
  29. # Set random seed for reproducibility
  30. np.random.seed(42)
  31. # Configure scanpy
  32. sc.settings.verbosity = 1
  33. sc.settings.set_figure_params(dpi=100, frameon=False, figsize=(6, 5))
  34. # %%
  35. # Constants (from constants.R)
  36. RESOLUTION = 2
  37. UMAP_SEED = 42
  38. UMAP_NEIGHBORS = 20
  39. UMAP_MIN_DIST = 0.2
  40. UMAP_METRIC = "cosine"
  41. UMAP_COMPONENTS = 10
  42. N_PCS = 100
  43. # %% [markdown]
  44. # ## 2. Data Loading
  45. #
  46. # Load the MATLAB data file and create a MuData object with multiple modalities.
  47. # %%
  48. def clr_normalize(X):
  49. """
  50. Centered Log-Ratio (CLR) normalization per cell (row).
  51. Equivalent to Seurat's NormalizeData(method='CLR', margin=4).
  52. For each cell, computes: log(x + 1) - mean(log(x + 1))
  53. This centers the log-transformed data around zero for each cell.
  54. """
  55. # Add pseudocount and log transform
  56. X_pseudo = X + 1
  57. log_X = np.log(X_pseudo)
  58. # Compute geometric mean per cell (row) and center
  59. geometric_mean = np.mean(log_X, axis=1, keepdims=True)
  60. X_clr = log_X - geometric_mean
  61. return X_clr
  62. # %%
  63. def extract_matlab_strings(arr):
  64. """
  65. Extract strings from MATLAB cell arrays loaded by scipy.io.loadmat.
  66. MATLAB cell arrays come as nested numpy arrays of objects.
  67. """
  68. arr = np.asarray(arr)
  69. # Flatten if needed
  70. if arr.ndim > 1:
  71. arr = arr.flatten()
  72. # If it's a single element containing the whole array as string, something is wrong
  73. # This happens when squeeze_me=True mangles cell arrays
  74. if len(arr) == 1 and isinstance(arr[0], str) and '[' in arr[0]:
  75. # This is a string representation of the array, not the actual array
  76. raise ValueError("MATLAB cell array was converted to string representation. Reload with squeeze_me=False")
  77. result = []
  78. for item in arr:
  79. if isinstance(item, np.ndarray):
  80. # Nested array - extract the string
  81. if item.size > 0:
  82. val = item.flat[0]
  83. result.append(str(val).strip())
  84. else:
  85. result.append('')
  86. elif isinstance(item, str):
  87. result.append(item.strip())
  88. else:
  89. result.append(str(item).strip())
  90. return np.array(result)
  91. def load_rda_file(filepath):
  92. """
  93. Load an R .Rda file and return the first object as a numpy array.
  94. """
  95. try:
  96. import pyreadr
  97. result = pyreadr.read_r(filepath)
  98. # Get the first (and usually only) object
  99. key = list(result.keys())[0]
  100. return result[key].values.flatten()
  101. except ImportError:
  102. print(f"Warning: pyreadr not installed. Cannot load {filepath}")
  103. print("Install with: pip install pyreadr")
  104. return None
  105. except Exception as e:
  106. print(f"Warning: Could not load {filepath}: {e}")
  107. return None
  108. def load_jianing_data(mat_file_path):
  109. """
  110. Load the Jianing mouse S1 dataset from MATLAB file.
  111. Creates a MuData object with the following modalities:
  112. - ISI: Inter-spike interval data (100 features)
  113. - WF: Waveform data (351 features)
  114. - PSTH: Peri-stimulus time histogram (151 features)
  115. - features: Hand-crafted features (7 features)
  116. Returns:
  117. mdata: MuData object with all modalities
  118. metadata_df: DataFrame with cell metadata
  119. """
  120. import os
  121. # Load MATLAB file - use squeeze_me=False to preserve array structure
  122. mat_data = loadmat(mat_file_path, squeeze_me=False, struct_as_record=False)
  123. # Extract dimensions
  124. depth_arr = np.asarray(mat_data['depth']).flatten()
  125. n_cells = len(depth_arr)
  126. cell_ids = [f"cell_{i}" for i in range(n_cells)]
  127. # Extract continuous variables
  128. depth = depth_arr.astype(np.float64)
  129. latency = np.asarray(mat_data['latency']).flatten().astype(np.float64)
  130. latency = np.nan_to_num(latency, nan=0.0)
  131. onset_rate = np.asarray(mat_data['onset_rate']).flatten().astype(np.float64)
  132. onset_rate = np.nan_to_num(onset_rate, nan=0.0)
  133. # Extract categorical variables (MATLAB cell arrays need special handling)
  134. layer = extract_matlab_strings(mat_data['layer'])
  135. cell_type = extract_matlab_strings(mat_data['true_cell_type'])
  136. # Create layerCellType (combined label)
  137. layer_cell_type = np.array([f"{ct}-{l}" for ct, l in zip(cell_type, layer)])
  138. # Extract feature matrices
  139. ISI = np.asarray(mat_data['ISI']).astype(np.float64)
  140. WF = np.asarray(mat_data['waveformV']).astype(np.float64)
  141. PSTH = np.asarray(mat_data['AllResp']).astype(np.float64)
  142. # Ensure correct orientation (cells x features)
  143. if ISI.shape[0] != n_cells:
  144. ISI = ISI.T
  145. if WF.shape[0] != n_cells:
  146. WF = WF.T
  147. if PSTH.shape[0] != n_cells:
  148. PSTH = PSTH.T
  149. # Extract additional features from 'features' matrix
  150. features_mat = np.asarray(mat_data['features']).astype(np.float64)
  151. if features_mat.shape[0] != n_cells:
  152. features_mat = features_mat.T
  153. features_mat = np.nan_to_num(features_mat, nan=0.0)
  154. # Load width and ratio_p2t from .Rda files (like the R code does)
  155. base_dir = os.path.dirname(mat_file_path)
  156. width_path = os.path.join(base_dir, 'width.Rda')
  157. width = load_rda_file(width_path)
  158. if width is None:
  159. print("Warning: Could not load width.Rda, using zeros")
  160. width = np.zeros(n_cells)
  161. else:
  162. width = width.astype(np.float64)
  163. print(f"Loaded width from .Rda: min={width.min():.4f}, max={width.max():.4f}")
  164. ratio_p2t_path = os.path.join(base_dir, 'ratio_p2t.Rda')
  165. ratio_p2t = load_rda_file(ratio_p2t_path)
  166. if ratio_p2t is None:
  167. print("Warning: Could not load ratio_p2t.Rda, using zeros")
  168. ratio_p2t = np.zeros(n_cells)
  169. else:
  170. ratio_p2t = ratio_p2t.astype(np.float64)
  171. print(f"Loaded ratio_p2t from .Rda: min={ratio_p2t.min():.4f}, max={ratio_p2t.max():.4f}")
  172. # Create metadata DataFrame
  173. metadata_df = pd.DataFrame({
  174. 'Layer': layer,
  175. 'CellType': cell_type,
  176. 'layerCellType': layer_cell_type,
  177. 'Depth': depth,
  178. 'latency': latency,
  179. 'onset': onset_rate,
  180. 'width': width,
  181. 'ratio_p2t': ratio_p2t
  182. }, index=cell_ids)
  183. # Create AnnData objects for each modality
  184. adata_isi = AnnData(
  185. X=ISI.astype(np.float32),
  186. obs=metadata_df.copy()
  187. )
  188. adata_isi.var_names = [f"ISI_{i}" for i in range(ISI.shape[1])]
  189. adata_wf = AnnData(
  190. X=WF.astype(np.float32),
  191. obs=metadata_df.copy()
  192. )
  193. adata_wf.var_names = [f"WF_{i}" for i in range(WF.shape[1])]
  194. adata_psth = AnnData(
  195. X=PSTH.astype(np.float32),
  196. obs=metadata_df.copy()
  197. )
  198. adata_psth.var_names = [f"PSTH_{i}" for i in range(PSTH.shape[1])]
  199. # Create features AnnData (hand-crafted features)
  200. feature_names = ['Depth', 'OnsetRate', 'Latency', 'StimFR', 'StimFF']
  201. if features_mat.shape[1] >= 7:
  202. feature_names.extend(['width', 'ratio_p2t'])
  203. else:
  204. # Add width and ratio_p2t to features matrix
  205. features_mat = np.column_stack([features_mat, width, ratio_p2t])
  206. feature_names.extend(['width', 'ratio_p2t'])
  207. adata_features = AnnData(
  208. X=features_mat.astype(np.float32),
  209. obs=metadata_df.copy()
  210. )
  211. adata_features.var_names = feature_names[:features_mat.shape[1]]
  212. # Create concatenated modality (WF + ISI + PSTH)
  213. concat_data = np.concatenate([WF, ISI, PSTH], axis=1)
  214. adata_concat = AnnData(
  215. X=concat_data.astype(np.float32),
  216. obs=metadata_df.copy()
  217. )
  218. concat_var_names = ([f"WF_{i}" for i in range(WF.shape[1])] +
  219. [f"ISI_{i}" for i in range(ISI.shape[1])] +
  220. [f"PSTH_{i}" for i in range(PSTH.shape[1])])
  221. adata_concat.var_names = concat_var_names
  222. # Create MuData object
  223. mdata = MuData({
  224. 'ISI': adata_isi,
  225. 'WF': adata_wf,
  226. 'PSTH': adata_psth,
  227. 'features': adata_features,
  228. 'concat': adata_concat
  229. })
  230. print(f"\nLoaded {n_cells} cells")
  231. print(f"Modalities:")
  232. print(f" - ISI: {adata_isi.shape}")
  233. print(f" - WF: {adata_wf.shape}")
  234. print(f" - PSTH: {adata_psth.shape}")
  235. print(f" - features: {adata_features.shape}")
  236. print(f" - concat: {adata_concat.shape}")
  237. print(f"\nCell types: {np.unique(cell_type)}")
  238. print(f"Layers: {np.unique(layer)}")
  239. return mdata, metadata_df
  240. # %%
  241. # Load the data
  242. mat_file = "JianingData/MergedData.mat"
  243. mdata, metadata_df = load_jianing_data(mat_file)
  244. # %%
  245. # Inspect the MuData structure
  246. print(mdata)
  247. # %% [markdown]
  248. # ## 3. Single-Modality Processing
  249. #
  250. # Define a function to process each modality individually, equivalent to `calcRepresentation()` in the R code.
  251. # %%
  252. def process_modality(adata, modality_name, n_pcs=30, normalize=True,
  253. n_neighbors=20, metric='cosine', random_state=42):
  254. """
  255. Process a single modality through the full Seurat-equivalent pipeline.
  256. Steps:
  257. 1. CLR normalization (optional)
  258. 2. Find variable features (using all features for electrophysiology data)
  259. 3. Scale data (z-score normalization)
  260. 4. PCA
  261. 5. Find neighbors
  262. 6. UMAP (2D)
  263. 7. Leiden clustering
  264. Args:
  265. adata: AnnData object
  266. modality_name: Name of the modality (for storing results)
  267. n_pcs: Number of principal components
  268. normalize: Whether to apply CLR normalization
  269. n_neighbors: Number of neighbors for kNN graph
  270. metric: Distance metric for neighbors
  271. random_state: Random seed for reproducibility
  272. Returns:
  273. adata: Processed AnnData object with embeddings and clusters
  274. """
  275. adata = adata.copy()
  276. # Store raw data
  277. adata.layers['raw'] = adata.X.copy()
  278. # 1. CLR normalization (if requested)
  279. if normalize:
  280. adata.X = clr_normalize(adata.X)
  281. adata.layers['normalized'] = adata.X.copy()
  282. # 2. For electrophysiology data, use all features as variable
  283. # (unlike scRNA-seq where we select highly variable genes)
  284. adata.var['highly_variable'] = True
  285. # 3. Scale data (z-score normalization)
  286. sc.pp.scale(adata, max_value=None)
  287. # 4. PCA
  288. n_pcs_actual = min(n_pcs, min(adata.n_obs, adata.n_vars) - 1)
  289. sc.tl.pca(adata, n_comps=n_pcs_actual, svd_solver='arpack', random_state=random_state)
  290. # 5. Find neighbors
  291. sc.pp.neighbors(adata, n_neighbors=n_neighbors, n_pcs=n_pcs_actual,
  292. metric=metric, random_state=random_state)
  293. # 6. UMAP
  294. sc.tl.umap(adata, random_state=random_state, min_dist=UMAP_MIN_DIST)
  295. # 7. Leiden clustering
  296. sc.tl.leiden(adata, resolution=RESOLUTION, random_state=random_state)
  297. # Rename clustering column for this modality
  298. adata.obs[f'{modality_name}_leiden'] = adata.obs['leiden']
  299. print(f"Processed {modality_name}:")
  300. print(f" - PCA components: {n_pcs_actual}")
  301. print(f" - Clusters found: {adata.obs['leiden'].nunique()}")
  302. return adata
  303. # %% [markdown]
  304. # ## 4. Process Each Modality
  305. # %%
  306. # Process ISI modality (with CLR normalization)
  307. adata_isi = process_modality(
  308. mdata['ISI'],
  309. 'ISI',
  310. n_pcs=N_PCS,
  311. normalize=True,
  312. metric=UMAP_METRIC
  313. )
  314. # %%
  315. # Process WF modality (without normalization, per R code)
  316. adata_wf = process_modality(
  317. mdata['WF'],
  318. 'WF',
  319. n_pcs=N_PCS,
  320. normalize=False, # WF is not normalized in the R code
  321. metric=UMAP_METRIC
  322. )
  323. # %%
  324. # Process PSTH modality (with CLR normalization)
  325. adata_psth = process_modality(
  326. mdata['PSTH'],
  327. 'PSTH',
  328. n_pcs=N_PCS,
  329. normalize=True,
  330. metric=UMAP_METRIC
  331. )
  332. # %%
  333. # Process concatenated modality (with CLR normalization)
  334. # adata_concat = process_modality(
  335. # mdata['concat'],
  336. # 'concat',
  337. # n_pcs=N_PCS,
  338. # normalize=True,
  339. # metric=UMAP_METRIC
  340. # )
  341. # %% [markdown]
  342. # ## 5. Individual Modality Visualizations
  343. # %%
  344. # Create UMAP plots for each modality colored by layerCellType
  345. fig, axes = plt.subplots(1, 3, figsize=(15, 5))
  346. modalities = [
  347. (adata_wf, 'WF'),
  348. (adata_isi, 'ISI'),
  349. (adata_psth, 'PSTH'),
  350. ]
  351. for ax, (adata, name) in zip(axes, modalities):
  352. sc.pl.umap(adata, color='layerCellType', ax=ax, show=False,
  353. title=name, legend_loc='none', frameon=False, size=100)
  354. plt.tight_layout()
  355. plt.show()
  356. # %%
  357. # Create UMAP plots colored by cluster ID for each modality
  358. fig, axes = plt.subplots(1, 3, figsize=(15, 5))
  359. modalities_for_clustering = [
  360. (adata_wf, 'WF', 'WF_leiden'),
  361. (adata_isi, 'ISI', 'ISI_leiden'),
  362. (adata_psth, 'PSTH', 'PSTH_leiden')
  363. ]
  364. for ax, (adata, name, cluster_key) in zip(axes, modalities_for_clustering):
  365. sc.pl.umap(adata, color=cluster_key, ax=ax, show=False,
  366. title=name, legend_loc='on data', frameon=False,size=100)
  367. plt.tight_layout()
  368. plt.show()
  369. # %% [markdown]
  370. # ## 6. Multi-Modal Integration
  371. #
  372. # Integrate the three modalities (WF, ISI, PSTH) using a weighted nearest neighbors approach similar to Seurat's FindMultiModalNeighbors.
  373. # %%
  374. # Update the MuData object with processed modalities
  375. mdata.mod['ISI'] = adata_isi
  376. mdata.mod['WF'] = adata_wf
  377. mdata.mod['PSTH'] = adata_psth
  378. # mdata.mod['concat'] = adata_concat
  379. # Update the MuData object
  380. mdata.update()
  381. # %%
  382. # Muon's multi-modal weighted nearest neighbors integration
  383. # This computes cell-specific weights for each modality based on how well
  384. # each modality captures the local neighborhood structure
  385. # First, ensure each modality has its PCA stored with a unique key
  386. # We need to copy the PCA embeddings to the MuData level
  387. for mod_name in ['WF', 'ISI', 'PSTH']:
  388. mod = mdata.mod[mod_name]
  389. # Store PCA in obsm with modality-specific key
  390. key = f'X_pca_{mod_name}'
  391. mdata.obsm[key] = mod.obsm['X_pca'][:, :N_PCS]
  392. print(f"Stored {key}: {mdata.obsm[key].shape}")
  393. print(f"\nAvailable representations in mdata.obsm: {list(mdata.obsm.keys())}")
  394. # %%
  395. # Compute multi-modal weighted nearest neighbors using Muon
  396. # This is similar to Seurat's FindMultiModalNeighbors
  397. # We only integrate WF, ISI, and PSTH (not features or concat)
  398. print("Computing multi-modal weighted nearest neighbors...")
  399. # Create a subset MuData with only the modalities we want to integrate
  400. mdata_wnn = MuData({
  401. 'WF': mdata.mod['WF'],
  402. 'ISI': mdata.mod['ISI'],
  403. 'PSTH': mdata.mod['PSTH']
  404. })
  405. # Run Muon's WNN on the subset
  406. mu.pp.neighbors(
  407. mdata_wnn,
  408. key_added='wnn',
  409. n_neighbors=UMAP_NEIGHBORS,
  410. n_bandwidth_neighbors=20,
  411. n_multineighbors=200,
  412. metric='euclidean',
  413. low_memory=False
  414. )
  415. print("WNN graph computed successfully!")
  416. # %%
  417. # Compute UMAP on the WNN graph
  418. mu.tl.umap(mdata_wnn, neighbors_key='wnn', random_state=UMAP_SEED, min_dist=UMAP_MIN_DIST)
  419. # Copy the WNN UMAP back to the main mdata object
  420. mdata.obsm['X_wnn_umap'] = mdata_wnn.obsm['X_umap'].copy()
  421. print(f"WNN UMAP computed: {mdata.obsm['X_wnn_umap'].shape}")
  422. # Check if modality weights were computed
  423. weight_keys = [k for k in mdata_wnn.obs.columns if 'weight' in k.lower()]
  424. if weight_keys:
  425. print(f"\nModality weights available: {weight_keys}")
  426. # Copy weights to main mdata
  427. for key in weight_keys:
  428. mdata.obs[key] = mdata_wnn.obs[key].values
  429. # %% [markdown]
  430. # ## 7. Final Clustering on Integrated Data
  431. # %%
  432. # Run Leiden clustering on the WNN neighbor graph
  433. # The WNN graph is stored at the MuData level in obsp, so we use scanpy directly
  434. # First, let's see what's available
  435. print("Available in mdata_wnn.obsp:", list(mdata_wnn.obsp.keys()))
  436. print("Available in mdata_wnn.uns:", list(mdata_wnn.uns.keys()))
  437. # The WNN connectivities should be in obsp with a key like 'wnn_connectivities'
  438. # We need to run leiden using scanpy on an AnnData that has the WNN graph
  439. # Create a temporary AnnData to run leiden on
  440. import anndata
  441. adata_for_clustering = anndata.AnnData(
  442. X=np.zeros((mdata_wnn.n_obs, 1)), # Dummy X matrix
  443. obs=mdata_wnn.obs.copy()
  444. )
  445. # Copy the WNN graph to the temporary AnnData
  446. connectivities_key = 'wnn_connectivities' if 'wnn_connectivities' in mdata_wnn.obsp else 'connectivities'
  447. distances_key = 'wnn_distances' if 'wnn_distances' in mdata_wnn.obsp else 'distances'
  448. adata_for_clustering.obsp['connectivities'] = mdata_wnn.obsp[connectivities_key]
  449. if distances_key in mdata_wnn.obsp:
  450. adata_for_clustering.obsp['distances'] = mdata_wnn.obsp[distances_key]
  451. # Add the neighbors info to uns
  452. adata_for_clustering.uns['neighbors'] = {
  453. 'connectivities_key': 'connectivities',
  454. 'distances_key': 'distances',
  455. 'params': {'n_neighbors': UMAP_NEIGHBORS, 'method': 'wnn'}
  456. }
  457. # Run Leiden clustering
  458. sc.tl.leiden(adata_for_clustering, resolution=RESOLUTION, random_state=UMAP_SEED)
  459. # Copy the clustering results back to main mdata
  460. mdata.obs['wnn_leiden'] = adata_for_clustering.obs['leiden'].values
  461. mdata_wnn.obs['leiden'] = adata_for_clustering.obs['leiden'].values
  462. print(f"Found {mdata.obs['wnn_leiden'].nunique()} clusters at resolution {RESOLUTION}")
  463. # %% [markdown]
  464. # ## 8. Final Visualizations
  465. # %%
  466. # Prepare data for plotting
  467. # Get UMAP coordinates
  468. umap_df = pd.DataFrame(
  469. mdata.obsm['X_wnn_umap'],
  470. columns=['UMAP1', 'UMAP2'],
  471. index=mdata.obs_names
  472. )
  473. # Get metadata from one of the modalities (they all have the same metadata)
  474. mod_obs = mdata.mod['WF'].obs
  475. umap_df['layerCellType'] = mod_obs['layerCellType'].values
  476. umap_df['CellType'] = mod_obs['CellType'].values
  477. umap_df['cluster'] = mdata.obs['wnn_leiden'].values
  478. # Copy additional metadata
  479. for col in ['Depth', 'latency', 'onset', 'width', 'ratio_p2t']:
  480. if col in mod_obs.columns:
  481. umap_df[col] = mod_obs[col].values
  482. # %%
  483. # Create combined visualization (similar to pComb in R)
  484. fig, axes = plt.subplots(1, 3, figsize=(18, 5))
  485. # Plot 1: WNN UMAP colored by layerCellType
  486. ax = axes[0]
  487. for label in umap_df['layerCellType'].unique():
  488. mask = umap_df['layerCellType'] == label
  489. ax.scatter(umap_df.loc[mask, 'UMAP1'], umap_df.loc[mask, 'UMAP2'],
  490. label=label, s=30, alpha=0.7)
  491. ax.set_xlabel('UMAP1')
  492. ax.set_ylabel('UMAP2')
  493. ax.set_title('WNN UMAP - Layer+CellType')
  494. ax.legend(bbox_to_anchor=(1.05, 1), loc='upper left', fontsize=8)
  495. ax.set_aspect('equal')
  496. # Plot 2: WNN UMAP colored by CellType
  497. ax = axes[1]
  498. for label in umap_df['CellType'].unique():
  499. mask = umap_df['CellType'] == label
  500. ax.scatter(umap_df.loc[mask, 'UMAP1'], umap_df.loc[mask, 'UMAP2'],
  501. label=label, s=30, alpha=0.7)
  502. ax.set_xlabel('UMAP1')
  503. ax.set_ylabel('UMAP2')
  504. ax.set_title('WNN UMAP - CellType')
  505. ax.legend(bbox_to_anchor=(1.05, 1), loc='upper left', fontsize=8)
  506. ax.set_aspect('equal')
  507. # Plot 3: WNN UMAP colored by cluster
  508. ax = axes[2]
  509. scatter = ax.scatter(umap_df['UMAP1'], umap_df['UMAP2'],
  510. c=umap_df['cluster'].astype(int), cmap='tab20', s=30, alpha=0.7)
  511. ax.set_xlabel('UMAP1')
  512. ax.set_ylabel('UMAP2')
  513. ax.set_title(f'WNN UMAP - Clusters (res={RESOLUTION})')
  514. ax.set_aspect('equal')
  515. # Add cluster labels at centroids
  516. for cluster in umap_df['cluster'].unique():
  517. mask = umap_df['cluster'] == cluster
  518. centroid_x = umap_df.loc[mask, 'UMAP1'].mean()
  519. centroid_y = umap_df.loc[mask, 'UMAP2'].mean()
  520. ax.annotate(str(cluster), (centroid_x, centroid_y), fontsize=10, fontweight='bold')
  521. plt.tight_layout()
  522. plt.show()
  523. # %%
  524. # Feature plots: Width and Peak-to-Trough ratio
  525. fig, axes = plt.subplots(1, 2, figsize=(14, 5))
  526. # Width plot
  527. ax = axes[0]
  528. # Convert width to milliseconds if needed
  529. width_ms = umap_df['width'] * 1000 if umap_df['width'].max() < 1 else umap_df['width']
  530. scatter = ax.scatter(umap_df['UMAP1'], umap_df['UMAP2'],
  531. c=width_ms, cmap='viridis', s=30, alpha=0.7)
  532. plt.colorbar(scatter, ax=ax, label='Width (ms)')
  533. ax.set_xlabel('UMAP1')
  534. ax.set_ylabel('UMAP2')
  535. ax.set_title('Width of Waveform')
  536. ax.set_aspect('equal')
  537. # Peak-to-trough ratio plot
  538. ax = axes[1]
  539. scatter = ax.scatter(umap_df['UMAP1'], umap_df['UMAP2'],
  540. c=umap_df['ratio_p2t'], cmap='viridis', s=30, alpha=0.7)
  541. plt.colorbar(scatter, ax=ax, label='Peak to Trough Ratio')
  542. ax.set_xlabel('UMAP1')
  543. ax.set_ylabel('UMAP2')
  544. ax.set_title('Peak to Trough')
  545. ax.set_aspect('equal')
  546. plt.tight_layout()
  547. plt.show()
  548. # %%
  549. # Latency plot
  550. fig, ax = plt.subplots(figsize=(8, 6))
  551. # Use log scale for latency
  552. log_latency = np.log1p(umap_df['latency'])
  553. scatter = ax.scatter(umap_df['UMAP1'], umap_df['UMAP2'],
  554. c=log_latency, cmap='viridis', s=30, alpha=0.7)
  555. plt.colorbar(scatter, ax=ax, label='Log(Latency + 1)')
  556. ax.set_xlabel('UMAP1')
  557. ax.set_ylabel('UMAP2')
  558. ax.set_title('Latency')
  559. ax.set_aspect('equal')
  560. plt.tight_layout()
  561. plt.show()
  562. # %%
  563. # Width vs Peak-to-Trough scatter plot with marginal histograms
  564. # (equivalent to ggscatterhist in R)
  565. # Prepare data with log transformations
  566. plot_df = umap_df.copy()
  567. plot_df['log_width'] = np.log(plot_df['width'] + 1e-6)
  568. plot_df['log_ratio_p2t'] = np.log(plot_df['ratio_p2t'] + 1e-6)
  569. # Create joint plot
  570. g = sns.jointplot(
  571. data=plot_df,
  572. x='log_width',
  573. y='log_ratio_p2t',
  574. hue='layerCellType',
  575. kind='scatter',
  576. height=8,
  577. alpha=0.7
  578. )
  579. g.ax_joint.set_xlabel('Log(Width)')
  580. g.ax_joint.set_ylabel('Log(Peak-to-Trough Ratio)')
  581. plt.suptitle('Width vs Peak-to-Trough Ratio by Cell Type', y=1.02)
  582. plt.show()
  583. # %% [markdown]
  584. # ## 9. Modal Weight Visualization
  585. #
  586. # Since we used equal weights for the simple concatenation approach, this section shows placeholder visualizations. For true WNN-style adaptive weights, consider implementing Seurat's WNN algorithm or using muon's multi-modal analysis tools.
  587. # %%
  588. # Use the modality weights computed by Muon's WNN
  589. # Muon stores them as 'WF:mod_weight', 'ISI:mod_weight', 'PSTH:mod_weight'
  590. print("Modality weights from Muon WNN:")
  591. print(f"Available weight columns: {[col for col in mdata_wnn.obs.columns if 'weight' in col.lower()]}")
  592. # Rename weights for easier access
  593. weight_mapping = {
  594. 'WF:mod_weight': 'WF_weight',
  595. 'ISI:mod_weight': 'ISI_weight',
  596. 'PSTH:mod_weight': 'PSTH_weight'
  597. }
  598. for old_name, new_name in weight_mapping.items():
  599. if old_name in mdata_wnn.obs.columns:
  600. mdata_wnn.obs[new_name] = mdata_wnn.obs[old_name]
  601. mdata.obs[new_name] = mdata_wnn.obs[old_name].values
  602. print(f"{new_name} - mean: {mdata_wnn.obs[old_name].mean():.3f}, std: {mdata_wnn.obs[old_name].std():.3f}")
  603. # Add weights to umap_df for plotting
  604. for weight_col in ['WF_weight', 'ISI_weight', 'PSTH_weight']:
  605. if weight_col in mdata.obs.columns:
  606. umap_df[weight_col] = mdata.obs[weight_col].values
  607. # %%
  608. # Visualize modality weights
  609. fig, axes = plt.subplots(1, 3, figsize=(18, 5))
  610. # Plot 1: Weight histograms (like p.hist in R)
  611. ax = axes[0]
  612. colors = {'WF_weight': '#F39922', 'ISI_weight': '#12A84B', 'PSTH_weight': '#0B78BE'}
  613. labels = {'WF_weight': 'WF', 'ISI_weight': 'ISI', 'PSTH_weight': 'PSTH'}
  614. for weight_col in ['WF_weight', 'ISI_weight', 'PSTH_weight']:
  615. if weight_col in umap_df.columns:
  616. ax.hist(umap_df[weight_col], bins=20, alpha=0.6,
  617. color=colors[weight_col], label=labels[weight_col])
  618. ax.set_xlabel('Weight')
  619. ax.set_ylabel('Count')
  620. ax.set_title('Modality Weight Distributions')
  621. ax.legend()
  622. # Plot 2: UMAP colored by dominant modality
  623. ax = axes[1]
  624. if all(col in umap_df.columns for col in ['WF_weight', 'ISI_weight', 'PSTH_weight']):
  625. # Determine dominant modality for each cell
  626. weight_cols = ['WF_weight', 'ISI_weight', 'PSTH_weight']
  627. dominant = umap_df[weight_cols].idxmax(axis=1)
  628. dominant_colors = dominant.map({'WF_weight': colors['WF_weight'],
  629. 'ISI_weight': colors['ISI_weight'],
  630. 'PSTH_weight': colors['PSTH_weight']})
  631. for mod, color in [('WF_weight', colors['WF_weight']),
  632. ('ISI_weight', colors['ISI_weight']),
  633. ('PSTH_weight', colors['PSTH_weight'])]:
  634. mask = dominant == mod
  635. ax.scatter(umap_df.loc[mask, 'UMAP1'], umap_df.loc[mask, 'UMAP2'],
  636. c=color, label=labels[mod], s=30, alpha=0.7)
  637. ax.set_xlabel('UMAP1')
  638. ax.set_ylabel('UMAP2')
  639. ax.set_title('Dominant Modality per Cell')
  640. ax.legend()
  641. ax.set_aspect('equal')
  642. # Plot 3: Stacked bar showing weight composition by cell type
  643. ax = axes[2]
  644. if 'CellType' in umap_df.columns:
  645. weight_by_type = umap_df.groupby('CellType')[['WF_weight', 'ISI_weight', 'PSTH_weight']].mean()
  646. weight_by_type.plot(kind='bar', stacked=True, ax=ax,
  647. color=[colors['WF_weight'], colors['ISI_weight'], colors['PSTH_weight']])
  648. ax.set_xlabel('Cell Type')
  649. ax.set_ylabel('Mean Weight')
  650. ax.set_title('Average Modality Weights by Cell Type')
  651. ax.legend(['WF', 'ISI', 'PSTH'], bbox_to_anchor=(1.02, 1), loc='upper left')
  652. ax.set_xticklabels(ax.get_xticklabels(), rotation=45, ha='right')
  653. plt.tight_layout()
  654. plt.show()
  655. # %% [markdown]
  656. # ## 10. Classification Analysis
  657. #
  658. # Compare classification accuracy between multimodal (WNN) and unimodal (WF, ISI, PSTH) representations for the five most prevalent cell types: E-4, E-5, FS-4, FS-5, SOM-nan.
  659. #
  660. # This mirrors the analysis in `classifyData.r` using scikit-learn's GradientBoostingClassifier.
  661. # %%
  662. from sklearn.ensemble import GradientBoostingClassifier
  663. from sklearn.model_selection import train_test_split
  664. from sklearn.metrics import balanced_accuracy_score
  665. def classify_embeddings(embeddings, labels, n_repeats=20, test_size=0.2):
  666. """
  667. Perform classification using GradientBoostingClassifier.
  668. Mirrors the R code in classifyData.r which uses:
  669. - method = "gbm" (Gradient Boosting Machine)
  670. - 80/20 train/test split
  671. - Multiple random seeds for robust evaluation
  672. Args:
  673. embeddings: numpy array of shape (n_samples, n_features)
  674. labels: numpy array of class labels
  675. n_repeats: number of times to repeat classification with different splits
  676. test_size: fraction of data to use for testing
  677. Returns:
  678. per_class_accuracies: dict mapping class -> list of balanced accuracies
  679. overall_accuracies: list of overall balanced accuracies (mean of per-class)
  680. classes: list of unique class names
  681. """
  682. classes = np.unique(labels)
  683. per_class_acc = {cls: [] for cls in classes}
  684. overall_acc = []
  685. for seed in range(1, n_repeats + 1):
  686. # Split data
  687. X_train, X_test, y_train, y_test = train_test_split(
  688. embeddings, labels, test_size=test_size,
  689. random_state=seed, stratify=labels
  690. )
  691. # Train GBM classifier
  692. clf = GradientBoostingClassifier(
  693. n_estimators=100,
  694. learning_rate=0.1,
  695. max_depth=3,
  696. random_state=seed
  697. )
  698. clf.fit(X_train, y_train)
  699. # Predict
  700. y_pred = clf.predict(X_test)
  701. # Calculate per-class balanced accuracy
  702. run_per_class_acc = []
  703. for cls in classes:
  704. # For each class, calculate balanced accuracy treating it as binary
  705. cls_mask = (y_test == cls)
  706. if cls_mask.sum() > 0:
  707. # True positive rate (sensitivity) for this class
  708. tp_rate = ((y_pred == cls) & (y_test == cls)).sum() / (y_test == cls).sum()
  709. # True negative rate (specificity) for this class
  710. tn_rate = ((y_pred != cls) & (y_test != cls)).sum() / (y_test != cls).sum()
  711. balanced_acc = (tp_rate + tn_rate) / 2 * 100
  712. per_class_acc[cls].append(balanced_acc)
  713. run_per_class_acc.append(balanced_acc)
  714. # Overall balanced accuracy = mean of per-class balanced accuracies
  715. # This matches R's confusionMatrix approach
  716. overall_bal_acc = np.mean(run_per_class_acc)
  717. overall_acc.append(overall_bal_acc)
  718. return per_class_acc, overall_acc, classes
  719. # %%
  720. # Filter for the 5 prevalent cell types used in the R analysis
  721. # These are: E-4, E-5, FS-4, FS-5, SOM-nan
  722. target_cell_types = ['E-4', 'E-5', 'FS-4', 'FS-5', 'SOM-nan']
  723. # Get layerCellType labels from modality metadata
  724. layer_cell_types = mdata.mod['WF'].obs['layerCellType'].values
  725. # Create boolean mask for cells matching target types
  726. mask = np.isin(layer_cell_types, target_cell_types)
  727. # Get filtered labels
  728. filtered_labels = layer_cell_types[mask]
  729. print(f"Total cells: {len(layer_cell_types)}")
  730. print(f"Cells in target types: {mask.sum()}")
  731. print(f"\nCell type distribution:")
  732. for ct in target_cell_types:
  733. count = (filtered_labels == ct).sum()
  734. print(f" {ct}: {count}")
  735. # %%
  736. # Compute higher-dimensional UMAP embeddings for classification
  737. # This matches R's UMAP.components in constants.R
  738. n_umap_dims = UMAP_COMPONENTS # Currently 10
  739. # Dynamic key name based on dimensions
  740. umap_key = f'X_umap_{n_umap_dims}d'
  741. wnn_umap_key = f'X_wnn_umap_{n_umap_dims}d'
  742. print(f"Computing {n_umap_dims}D UMAP embeddings for classification...")
  743. print(f"Storing in keys: '{umap_key}' and '{wnn_umap_key}'")
  744. # WF
  745. sc.tl.umap(adata_wf, n_components=n_umap_dims, random_state=UMAP_SEED, min_dist=UMAP_MIN_DIST)
  746. adata_wf.obsm[umap_key] = adata_wf.obsm['X_umap'].copy()
  747. print(f" WF: {adata_wf.obsm[umap_key].shape}")
  748. # ISI
  749. sc.tl.umap(adata_isi, n_components=n_umap_dims, random_state=UMAP_SEED, min_dist=UMAP_MIN_DIST)
  750. adata_isi.obsm[umap_key] = adata_isi.obsm['X_umap'].copy()
  751. print(f" ISI: {adata_isi.obsm[umap_key].shape}")
  752. # PSTH
  753. sc.tl.umap(adata_psth, n_components=n_umap_dims, random_state=UMAP_SEED, min_dist=UMAP_MIN_DIST)
  754. adata_psth.obsm[umap_key] = adata_psth.obsm['X_umap'].copy()
  755. print(f" PSTH: {adata_psth.obsm[umap_key].shape}")
  756. # WNN - on the combined neighbor graph
  757. mu.tl.umap(mdata_wnn, neighbors_key='wnn', n_components=n_umap_dims,
  758. random_state=UMAP_SEED, min_dist=UMAP_MIN_DIST)
  759. mdata.obsm[wnn_umap_key] = mdata_wnn.obsm['X_umap'].copy()
  760. print(f" WNN: {mdata.obsm[wnn_umap_key].shape}")
  761. print("\nUMAP embeddings computed!")
  762. # %%
  763. # Run classification on each modality's UMAP embeddings
  764. n_repeats = 20
  765. # Prepare UMAP embeddings for each modality (using dynamic keys from previous cell)
  766. embeddings_dict = {
  767. 'WNN': mdata.obsm[wnn_umap_key][mask],
  768. 'WF': adata_wf.obsm[umap_key][mask],
  769. 'ISI': adata_isi.obsm[umap_key][mask],
  770. 'PSTH': adata_psth.obsm[umap_key][mask],
  771. }
  772. # Add features (width, ratio_p2t) if available
  773. if 'width' in mod_obs.columns and 'ratio_p2t' in mod_obs.columns:
  774. width_vals = mod_obs['width'].values[mask]
  775. ratio_vals = mod_obs['ratio_p2t'].values[mask]
  776. # Replace NaN with 0
  777. width_vals = np.nan_to_num(width_vals, nan=0.0)
  778. ratio_vals = np.nan_to_num(ratio_vals, nan=0.0)
  779. embeddings_dict['Features'] = np.column_stack([width_vals, ratio_vals])
  780. print(f"Using {n_umap_dims}D UMAP embeddings for classification")
  781. print(f"Embedding dimensions:")
  782. for name, emb in embeddings_dict.items():
  783. print(f" {name}: {emb.shape}")
  784. # Run classification for each modality
  785. results = {}
  786. print("\nRunning classification analysis (this may take a few minutes)...\n")
  787. for mod_name, embeddings in embeddings_dict.items():
  788. print(f"Classifying {mod_name}...", end=" ")
  789. per_class_acc, overall_acc, classes = classify_embeddings(
  790. embeddings, filtered_labels, n_repeats=n_repeats
  791. )
  792. results[mod_name] = {
  793. 'per_class': per_class_acc,
  794. 'overall': overall_acc,
  795. 'classes': classes
  796. }
  797. print(f"Mean accuracy: {np.mean(overall_acc):.1f}% ± {np.std(overall_acc):.1f}%")
  798. print("\nClassification complete!")
  799. # %%
  800. # Create summary DataFrame for plotting
  801. summary_data = []
  802. for mod_name, res in results.items():
  803. for cls in res['classes']:
  804. acc_values = res['per_class'][cls]
  805. summary_data.append({
  806. 'Modality': mod_name,
  807. 'CellType': cls,
  808. 'Accuracy': np.mean(acc_values),
  809. 'Std': np.std(acc_values),
  810. 'SE': np.std(acc_values) / np.sqrt(len(acc_values))
  811. })
  812. summary_df = pd.DataFrame(summary_data)
  813. # Also create raw data for statistical tests
  814. raw_data = []
  815. for mod_name, res in results.items():
  816. for cls in res['classes']:
  817. for run_idx, acc in enumerate(res['per_class'][cls]):
  818. raw_data.append({
  819. 'Run': run_idx + 1,
  820. 'Modality': mod_name,
  821. 'CellType': cls,
  822. 'Accuracy': acc
  823. })
  824. raw_df = pd.DataFrame(raw_data)
  825. print("Summary statistics:")
  826. print(summary_df.pivot_table(index='CellType', columns='Modality', values='Accuracy').round(1))
  827. # %%
  828. # Plot classification accuracy by cell type and modality
  829. # Similar to the R plot in classifyData.r
  830. fig, ax = plt.subplots(figsize=(12, 6))
  831. # Define colors and markers for each modality
  832. modality_colors = {
  833. 'WNN': '#E41A1C', # Red
  834. 'WF': '#377EB8', # Blue
  835. 'ISI': '#4DAF4A', # Green
  836. 'PSTH': '#984EA3', # Purple
  837. 'Features': '#FF7F00' # Orange
  838. }
  839. modality_order = ['WNN', 'WF', 'ISI', 'PSTH', 'Features']
  840. cell_types = sorted(summary_df['CellType'].unique())
  841. # Plot with dodge (offset) for each modality
  842. width = 0.15
  843. x = np.arange(len(cell_types))
  844. for i, mod in enumerate(modality_order):
  845. if mod not in results:
  846. continue
  847. mod_data = summary_df[summary_df['Modality'] == mod].set_index('CellType')
  848. # Reorder to match cell_types
  849. means = [mod_data.loc[ct, 'Accuracy'] if ct in mod_data.index else 0 for ct in cell_types]
  850. sems = [mod_data.loc[ct, 'SE'] if ct in mod_data.index else 0 for ct in cell_types]
  851. offset = (i - len(modality_order)/2 + 0.5) * width
  852. ax.errorbar(x + offset, means, yerr=sems,
  853. fmt='o-', color=modality_colors[mod],
  854. label=mod, capsize=3, markersize=8, linewidth=2)
  855. ax.set_xticks(x)
  856. ax.set_xticklabels(cell_types, rotation=45, ha='right')
  857. ax.set_xlabel('Cell Type')
  858. ax.set_ylabel('Classification Accuracy (%)')
  859. ax.set_title(f'Classification Accuracy at Embedding-D = {n_umap_dims}')
  860. ax.legend(title='Modality', bbox_to_anchor=(1.02, 1), loc='upper left')
  861. ax.set_ylim(0, 105)
  862. ax.axhline(y=100, color='gray', linestyle='--', alpha=0.3)
  863. ax.grid(axis='y', alpha=0.3)
  864. plt.tight_layout()
  865. plt.show()
  866. # %%
  867. # Overall accuracy comparison (barplot with error bars)
  868. fig, ax = plt.subplots(figsize=(8, 5))
  869. modalities = []
  870. means = []
  871. sems = []
  872. for mod in modality_order:
  873. if mod in results:
  874. modalities.append(mod)
  875. means.append(np.mean(results[mod]['overall']))
  876. sems.append(np.std(results[mod]['overall']) / np.sqrt(len(results[mod]['overall'])))
  877. x = np.arange(len(modalities))
  878. colors = [modality_colors[m] for m in modalities]
  879. bars = ax.bar(x, means, yerr=sems, capsize=5, color=colors, alpha=0.8, edgecolor='black')
  880. ax.set_xticks(x)
  881. ax.set_xticklabels(modalities)
  882. ax.set_ylabel('Overall Balanced Accuracy (%)')
  883. ax.set_title('Overall Classification Accuracy by Modality')
  884. ax.set_ylim(0, 100)
  885. ax.axhline(y=20, color='gray', linestyle='--', alpha=0.5, label='Chance level (5 classes)')
  886. # Add value labels on bars
  887. for bar, mean, sem in zip(bars, means, sems):
  888. ax.text(bar.get_x() + bar.get_width()/2, bar.get_height() + sem + 2,
  889. f'{mean:.1f}%', ha='center', va='bottom', fontsize=10)
  890. plt.tight_layout()
  891. plt.show()
  892. # Print summary
  893. print("\nOverall Accuracy Summary:")
  894. print("-" * 40)
  895. for mod in modality_order:
  896. if mod in results:
  897. acc = results[mod]['overall']
  898. print(f"{mod:10s}: {np.mean(acc):5.1f}% ± {np.std(acc):4.1f}%")
  899. # %% [markdown]
  900. # # Print summary
  901. # print("=" * 50)
  902. # print("Analysis Summary")
  903. # print("=" * 50)
  904. # print(f"\nTotal cells: {mdata.n_obs}")
  905. # print(f"\nModalities processed:")
  906. # print(f" - ISI: {mdata.mod['ISI'].n_vars} features")
  907. # print(f" - WF: {mdata.mod['WF'].n_vars} features")
  908. # print(f" - PSTH: {mdata.mod['PSTH'].n_vars} features")
  909. # print(f"\nWNN UMAP dimensions: {mdata.obsm['X_wnn_umap'].shape}")
  910. # print(f"\nClusters found: {mdata.obs['wnn_leiden'].nunique()}")
  911. #
  912. # # Get metadata from modality
  913. # mod_obs = mdata.mod['WF'].obs
  914. # print(f"\nCell types:")
  915. # print(mod_obs['CellType'].value_counts())
  916. # print(f"\nLayer distribution:")
  917. # print(mod_obs['Layer'].value_counts())
  918. #
  919. # # Print weight summary if available
  920. # if 'WF_weight' in mdata.obs.columns:
  921. # print(f"\nModality weight summary:")
  922. # for mod in ['WF', 'ISI', 'PSTH']:
  923. # col = f'{mod}_weight'
  924. # if col in mdata.obs.columns:
  925. # print(f" - {mod}: mean={mdata.obs[col].mean():.3f}, std={mdata.obs[col].std():.3f}")
  926. # %%
  927. # Print summary
  928. print("=" * 50)
  929. print("Analysis Summary")
  930. print("=" * 50)
  931. print(f"\nTotal cells: {mdata.n_obs}")
  932. print(f"\nModalities processed:")
  933. print(f" - ISI: {mdata.mod['ISI'].n_vars} features")
  934. print(f" - WF: {mdata.mod['WF'].n_vars} features")
  935. print(f" - PSTH: {mdata.mod['PSTH'].n_vars} features")
  936. # print(f"\nJoint embedding dimensions: {mdata.obsm['X_joint_pca'].shape[1]}")
  937. print(f"\nClusters found: {mdata.obs['wnn_leiden'].nunique()}")
  938. # print(f"\nCell types:")
  939. # print(mdata.obs['CellType'].value_counts())
  940. # print(f"\nLayer distribution:")
  941. # print(mdata.obs['Layer'].value_counts())
  942. # %%
  943. # Save processed data
  944. # mdata.write('processed_jianing.h5mu')
  945. # print("Saved processed data to processed_jianing.h5mu")
  946. # %%
  947. # %%

processJianing.ipynb at commit 08819f4, no license · at the source

Overview

13 affiliations
  1. Department of Psychological and Brain Sciences, Boston University, Boston, MA USA
  2. Department of Psychology, Boğaziçi University, Beşiktaş, Istanbul Turkey
  3. Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA USA
  4. Department of Biology, University of Oregon, Eugene, OR USA
  5. Institute of Neuroscience, University of Oregon, Eugene, OR USA
  6. Allen Institute for Neural Dynamics, Seattle, WA USA
  7. Department of Electrical Engineering, Columbia University, New York City, NY USA
  8. Zuckerman Institute, Columbia University, New York City, NY USA
  9. Department of Neurobiology and Biophysics, University of Washington, Seattle, WA USA
  10. Faculty of Computing & Data Sciences, Boston University, Boston, MA USA
  11. Center for Systems Neuroscience, Boston University, Boston, MA USA
  12. Department of Biomedical Engineering, Boston University, Boston, MA USA
  13. Department of Anatomy & Neurobiology, Boston University, Boston, MA USA
Institutions: Boston University (United States); Boğaziçi University (Türkiye); Massachusetts Institute of Technology (United States); University of Oregon (United States); Allen Institute for Neural Dynamics (United States); Mortimer B. Zuckerman Mind Brain Behavior Institute (United States); Columbia University (United States); University of Washington (United States)
Journal: Nature communications, volume 17, issue 1, article 5241
Dates: received 29 September 2025; accepted 19 March 2026; published online 15 April 2026
Type: Research article · Language: English
License: CC BY-NC-ND
Identifiers: DOI 10.1038/s41467-026-71331-0 · PMID 41986318 · PMCID PMC13260831 · OpenAlex W7154465353
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: genetics / omics (modality), extracellular electrophysiology (units, LFP) (modality), mouse (organism)
Methods: Spectral & time-frequency, Statistics, Smoothing, state filtering, decompositions, Machine learning, Evoked potentials, Graphs, fMRI & imaging, Single-unit activity, calcium imaging
Keywords: Machine learning, Computational neuroscience, Extracellular recording, Functional clustering
MeSH: Electrophysiological Phenomena*, Neurons*, Action Potentials, Animals, Mice, Multiomics (* major topic)
Topic: Cardiac electrophysiology and arrhythmias (Cardiology and Cardiovascular Medicine, Medicine), according to OpenAlex
Funding: NINDS NIH HHS (R01 NS131993, RF1 NS131993); U.S. Department of Health &amp; Human Services | NIH | National Institute of Neurological Disorders and Stroke (U01NS113252, R01NS122969, RF1NS131993, R00NS092972, R01NS121409, F31NS131018, R21NS135361); U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) (R01NS122969, RF1NS131993, R21NS135361, R01NS121409, R00NS092972, U01NS113252, F31NS131018); Whitehall Foundation (2019-12-77); Brain and Behavior Research Foundation (27923); U.S. Department of Health & Human Services | NIH | National Institute on Deafness and Other Communication Disorders (NIDCD) (R01DC01553); U.S. Department of Health &amp; Human Services | NIH | National Institute on Deafness and Other Communication Disorders (R01DC01553)
Citations: cited by 2 papers (Europe PMC); 105 references in the paper

Abstract

The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.

Repositories

Its files are read in the Code ↔ Paper reader above, with 8 matches between paragraphs and lines of code.

EricKenjiLee/PhysMAP_Manuscript

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 08819f45c7742001a760096c64bb00109bd8062e, 11 February 2026
Languages: R (17), Python (3), Jupyter (2), MATLAB (1)
Size: 77 files, 23 scripts
Software Heritage: not archived
Found in: “Code availability”
Holds: README, environment (poetry.lock, pyproject.toml, juxtacellular/requirements.txt), 2 notebooks
Not found: license file, CITATION.cff, tests, continuous integration, documentation
Tools: caret (9 files), reshape2 (9 files), tidyverse (8 files), NumPy (5 files), pandas (5 files), Matplotlib (4 files), seaborn (3 files), anndata (2 files), ggplot2 (2 files), ggpubr (2 files), reticulate (2 files), Scanpy (2 files), scikit-learn (2 files), SciPy (2 files), Seurat (2 files), cowplot (1 file), PyTorch (1 file), UMAP (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
24 files

Zenodo 18239113

License: CC-BY-4.0
State: the link answers, verified on 29 September 2026
Evidence: files inventoried
Size: 1 file
Software Heritage: not checked
Found in: “Code availability”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Availability: 1 check, the latest on 29 September 2026: the link answers (HTTP 200)
  • 29 September 2026: the link answers (HTTP 200)
At the source:

Code availability statement

The paper has a code availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:

Read it in the paper: doi.org/10.1038/s41467-026-71331-0.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 23 scripts, each with its path and the digest of its content;
  • 8 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Code and data availability statement

The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:

Read it in the paper: doi.org/10.1038/s41467-026-71331-0.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 29 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 12 authors, 4 keywords, 6 MeSH terms, 7 funders, 92 references.

Cite

This paper

Lee, E. K., Gül, A. E., Heller, G., Lakunina, A., Yu, H., Shelton, A., Olsen, S., Steinmetz, N. A., Hurwitz, C., Jaramillo, S., Przytycki, P. F., & Chandrasekaran, C. (2026). A multimodal approach for visualizing and identifying electrophysiological cell types in vivo. Nature communications, 17(1), 5241. https://doi.org/10.1038/s41467-026-71331-0

BibTeX

@article{lee2026multimodal,
author = {Lee, Eric Kenji and Gül, Asım E and Heller, Greggory and Lakunina, Anna and Yu, Han and Shelton, Andrew and Olsen, Shawn and Steinmetz, Nicholas A and Hurwitz, Cole and Jaramillo, Santiago and Przytycki, Pawel F and Chandrasekaran, Chandramouli},
title = {{A multimodal approach for visualizing and identifying electrophysiological cell types in vivo}},
journal = {Nature communications},
year = {2026},
month = apr,
volume = {17},
number = {1},
pages = {5241},
publisher = {Nature Publishing Group},
issn = {2041-1723},
doi = {10.1038/s41467-026-71331-0},
url = {https://doi.org/10.1038/s41467-026-71331-0},
pmid = {41986318},
pmcid = {PMC13260831}
}

RIS

TY - JOUR
AU - Lee, Eric Kenji
AU - Gül, Asım E
AU - Heller, Greggory
AU - Lakunina, Anna
AU - Yu, Han
AU - Shelton, Andrew
AU - Olsen, Shawn
AU - Steinmetz, Nicholas A
AU - Hurwitz, Cole
AU - Jaramillo, Santiago
AU - Przytycki, Pawel F
AU - Chandrasekaran, Chandramouli
TI - A multimodal approach for visualizing and identifying electrophysiological cell types in vivo
T2 - Nature communications
J2 - Nat Commun
PY - 2026
DA - 2026/04/15
VL - 17
IS - 1
SP - 5241
SN - 2041-1723
PB - Nature Publishing Group
DO - 10.1038/s41467-026-71331-0
UR - https://doi.org/10.1038/s41467-026-71331-0
LA - en
ER -

CSL-JSON

{
"id": "10.1038/s41467-026-71331-0",
"type": "article-journal",
"title": "A multimodal approach for visualizing and identifying electrophysiological cell types in vivo",
"container-title": "Nature communications",
"author": [
{
"family": "Lee",
"given": "Eric Kenji"
},
{
"family": "Gül",
"given": "Asım E"
},
{
"family": "Heller",
"given": "Greggory"
},
{
"family": "Lakunina",
"given": "Anna"
},
{
"family": "Yu",
"given": "Han"
},
{
"family": "Shelton",
"given": "Andrew"
},
{
"family": "Olsen",
"given": "Shawn"
},
{
"family": "Steinmetz",
"given": "Nicholas A"
},
{
"family": "Hurwitz",
"given": "Cole"
},
{
"family": "Jaramillo",
"given": "Santiago"
},
{
"family": "Przytycki",
"given": "Pawel F"
},
{
"family": "Chandrasekaran",
"given": "Chandramouli"
}
],
"container-title-short": "Nat Commun",
"volume": "17",
"issue": "1",
"page": "5241",
"DOI": "10.1038/s41467-026-71331-0",
"PMID": "41986318",
"PMCID": "PMC13260831",
"ISSN": "2041-1723",
"publisher": "Nature Publishing Group",
"URL": "https://doi.org/10.1038/s41467-026-71331-0",
"language": "en",
"issued": {
"date-parts": [
[
2026,
4,
15
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-76939-w [code]
HIPPIE: a generative model for electrophysiological analysis across species, technologies, and modalities.
Journal: Nature communications
In common: caret, reticulate, UMAP, 15 other tools, mouse, 10 references
[2] doi:10.1038/s41592-026-03076-z [code]
Neuropixels Opto: combining high-resolution electrophysiology and optogenetics.
Journal: Nature methods
In common: PyTorch, pandas, SciPy, 2 other tools, extracellular electrophysiology (units, LFP), mouse, 5 references, 2 authors
[3] doi:10.1038/s41586-026-10629-x [code]
Whole-genome duplication shaped cell-type evolution in the vertebrate brain.
Journal: Nature
In common: reticulate, UMAP, anndata, 13 other tools, genetics / omics, mouse
[4] doi:10.1016/j.xcrm.2026.102651 [code]
Integrative CSF profiling identifies disease-specific immune responses in leptomeningeal disease.
Journal: Cell reports. Medicine
In common: reticulate, UMAP, anndata, 13 other tools, genetics / omics
[5] doi:10.1016/j.xcrm.2026.102766 [code]
A longitudinal single-cell and spatial multiomic atlas of pediatric high-grade glioma.
Journal: Cell reports. Medicine
In common: UMAP, anndata, Scanpy, 12 other tools, genetics / omics, 2 references
[6] doi:10.1186/s13059-026-04177-w [code]
Genomic sequence evolution underlying human neocortical interareal diversification.
Journal: Genome biology
In common: reticulate, UMAP, anndata, 12 other tools, genetics / omics, mouse, 1 reference
[7] doi:10.1038/s41586-026-10490-y [code]
Lineage and organ signals sequentially build organ intrinsic nervous systems.
Journal: Nature
In common: UMAP, anndata, Scanpy, 12 other tools, mouse, 2 references
[8] doi:10.1186/s13073-026-01704-z [code]
Gene expression profiling enables refined parcellation of cortical layers in the heterogeneous human cerebral cortex.
Journal: Genome medicine
In common: UMAP, anndata, Scanpy, 13 other tools, genetics / omics, mouse
[9] doi:10.1038/s44320-026-00208-7 [code]
Interpretable deep generative ensemble learning for single-cell omics with Hydra.
Journal: Molecular systems biology
In common: reticulate, UMAP, anndata, 12 other tools, 1 reference
[10] doi:10.21203/rs.3.rs-9676637/v1 [code]
A Comprehensive Benchmarking of Spatial Deconvolution and Domain Detection Methods across Diverse Tissues and Spatial Transcriptomic Technologies
Journal: Research Square (preprint)
In common: reticulate, UMAP, anndata, 12 other tools, genetics / omics, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.