OSCR

Diminished Signal-to-Noise Ratio Disrupts Somatosensory Population Encoding and Drives Tactile Hyposensitivity in the Fmr1<sup>-/y</sup> Autism Model.

Code ↔ Paper

14 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 14 matches
  1. [1] § Materials and Methods › Decoding of Behavioral Responses ↔ Figures/response_decoding.py, lines 517–638 · score 0.85 · double cross validation, Classification accuracy, logistic regression, undersampling, penalty, SEM
  2. [2] § Materials and Methods › Statistics ↔ percephone/plts/utils.py, lines 41–102 · score 0.76 · Wilcoxon Signed Rank, Shapiro Wilk, Mann Whitney, Variance
  3. [3] § Results › Neuronal Activity in the S1‐FP Fails to Predict Stimulus Detection in Fmr1−/y‐Hyposensitive Mice ↔ Figures/response_decoding.py, lines 799–919 · score 0.75 · Wilcoxon signed rank, Miss accuracy, Hit accuracy, Response decoding, horizontal, Vertical
  4. [4] § Results › Forepaw‐Based Perceptual Decision‐Making Task Assesses Tactile Detection in Mice ↔ Figures/noise_assessment.py, lines 2771–2895 · score 0.73 · Correct Rejection, stimulus onset, Pre stimulus, behavioral outcomes, CR, Go trials
  5. [5] § Results › S1‐FP Pyramidal Neurons Dominate the Decoding of Stimulus Detection ↔ Figures/response_decoding.py, lines 1045–1134 · score 0.73 · trained logistic regression, Decoding accuracy, scored activity, stimulus detection, classifiers, model
  6. [6] § Results › Reducing Neuronal Excitability Improves Tactile Sensitivity in Fmr1−/y Mice ↔ Figures/response_decoding.py, lines 799–919 · score 0.67 · Miss classification, Miss accuracy, Hit accuracy, Response decoding, shuffled, post
  7. [7] § Results › Reduced Population Signal‐to‐Noise Ratio is Linked to Impaired Stimulus and Detection Encoding in Fmr1−/y‐Hyposensitive Mice ↔ Figures/noise_assessment.py, lines 2990–3119 · score 0.67 · stimulus evoked activity, population SNR, noise ratio, Go trials, quantified, signal
  8. [8] § Materials and Methods › Statistics ↔ percephone/plts/stats.py, lines 707–760 · score 0.64 · Shapiro Wilk, Mann Whitney, Friedman, ANOVA
  9. [9] § Materials and Methods › Decoding of Behavioral Responses ↔ Figures/response_decoding.py, lines 1045–1134 · score 0.63 · compare decoding accuracy, cross validation, fold, excitatory, trained, INH
  10. [10] § Materials and Methods › Go/No‐Go Vibrotactile Task Analysis ↔ percephone/plts/behavior.py, lines 608–660 · score 0.59 · Psychometric curves, Hit rate, Detection thresholds, sigmoid, fitted, amplitude
  11. [11] § Results › Neuronal Activity in the S1‐FP Fails to Predict Stimulus Detection in Fmr1−/y‐Hyposensitive Mice ↔ Figures/stimulus_encoding.py, lines 1044–1145 · score 0.57 · Post hoc, stimuli encoded, Neuronal activity, excitatory, stimulation, inhibitory
  12. [12] § Results › Forepaw‐Based Perceptual Decision‐Making Task Assesses Tactile Detection in Mice ↔ Figures/stimulus_encoding.py, lines 800–884 · score 0.56 · Correct Rejection, stimulus onset, CR, Go trials, FA, Alarm
  13. [13] § Materials and Methods › Analysis ↔ Figures/noise_assessment.py, lines 2584–2656 · score 0.56 · GABAergic, pyramidal neurons, Inhibitory neurons, interneurons, scores, excitatory
  14. [14] § Results › Reduced Single‐Neuron Signal‐to‐Noise Ratio Underlies Diminished Neural Recruitment in Fmr1−/y‐Hyposensitive Mice ↔ Figures/noise_assessment.py, lines 2584–2656 · score 0.53 · score response, GABAergic, pyramidal neurons, noise, Linear, interneurons

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 1,542 lines · 88 KB · GPL-2.0 · 5 matches

  1. # region ======================================== Imports ==============================================================
  2. import os
  3. import random
  4. import mplcursors
  5. import numpy as np
  6. import pandas as pd
  7. import statsmodels.formula.api as smf
  8. import statsmodels.api as sm
  9. import pingouin as pg
  10. from imblearn.under_sampling import RandomUnderSampler
  11. from matplotlib import pyplot as plt
  12. from multiprocessing import cpu_count, pool
  13. from scipy.stats import pointbiserialr, linregress, binomtest, wilcoxon
  14. from sklearn.linear_model import LogisticRegression
  15. from sklearn.metrics import confusion_matrix, classification_report
  16. from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
  17. from imblearn.pipeline import Pipeline
  18. from itertools import product
  19. from tqdm import tqdm
  20. import percephone.core.recording as pc
  21. import percephone.plts.stats as ppt
  22. import percephone.plts.style as sty
  23. # from Figures.noise_assessment import get_mean_trial_activity_df, ntn_cosine_similarity
  24. from Figures.stimulus_encoding import get_features
  25. # endregion
  26. def get_activity_by_frame_df(recs, zscore=True, BMS=False):
  27. """
  28. Builds a trial-by-trial DataFrame containing neuronal activity traces aligned to stimulus onset.
  29. For each neuron and trial, the function extracts activity values from 30 frames before to 30 frames
  30. after the stimulus (with frame 30 corresponding to the stimulus onset). Metadata such as genotype,
  31. recording ID, neuron type, trial characteristics, and neuron responsivity are also stored.
  32. If `zscore=True`, activity traces are taken from z-scored signals (`rec.zscore_exc` and `rec.zscore_inh`);
  33. otherwise, ΔF/F traces are used (`rec.df_f_exc` and `rec.df_f_inh`). When `BMS=True`, the recording ID
  34. includes the second part of the genotype string.
  35. Parameters
  36. ----------
  37. recs : list
  38. List of recording objects, each containing neuronal activity arrays and metadata
  39. (e.g., filename, genotype, stim_time, stim_ampl, stim_durations, detected_stim, session_threshold).
  40. zscore : bool, optional
  41. Whether to use z-scored activity (`True`) or raw ΔF/F traces (`False`). Default is True.
  42. BMS : bool, optional
  43. Whether to append the second part of the genotype string to the recording ID. Default is False.
  44. Returns
  45. -------
  46. pandas.DataFrame
  47. Long-format DataFrame where each row corresponds to the activity of a single neuron
  48. during a single trial. Columns include:
  49. - "Genotype": genotype of the recording,
  50. - "ID": recording identifier,
  51. - "Threshold": session threshold,
  52. - "Trial": trial index,
  53. - "Amplitude": stimulus amplitude,
  54. - "Duration": stimulus duration,
  55. - "Behavior": behavioral response label,
  56. - "n_type": neuron type ("EXC" or "INH"),
  57. - "resp": responsivity label of the neuron in that trial,
  58. - "n_ID": neuron index,
  59. - integer frame columns from -30 to +29 relative to stimulus onset.
  60. """
  61. rows = []
  62. for rec in recs:
  63. if BMS:
  64. rec_id = f"{rec.filename}-{rec.genotype.split("-")[1]}"
  65. else:
  66. rec_id = rec.filename
  67. activity_vector = [rec.zscore_exc, rec.zscore_inh] if zscore else [rec.df_f_exc, rec.df_f_inh]
  68. for n_type, activity in zip(["EXC", "INH"], activity_vector):
  69. resp = np.array(rec.matrices[n_type]["Responsivity"])
  70. for neuron_id, neuron in enumerate(np.array(activity)):
  71. for (trial_id, trial_time), trial_duration, trial_amp, trial_label in zip(enumerate(rec.stim_time),
  72. rec.stim_durations,
  73. rec.stim_ampl, rec.detected_stim):
  74. row = {"Genotype": rec.genotype, "ID": rec_id, "Threshold": rec.session_threshold,
  75. "Trial": trial_id, "Amplitude": trial_amp, "Duration": trial_duration,
  76. "Behavior": trial_label, "n_type": n_type, "resp": resp[neuron_id, trial_id], "n_ID": neuron_id}
  77. for frame_id, frame in enumerate(range(trial_time - 30, trial_time + 30)):
  78. row[frame_id] = neuron[frame]
  79. rows.append(row)
  80. return pd.DataFrame(rows)
  81. # Not used in the final paper
  82. def sliding_window_average(df, header_cols, window_size, sum=False):
  83. """
  84. Applies a sliding window operation across the numeric columns of a DataFrame, either averaging
  85. or summing values depending on the `sum` flag. The numeric columns are assumed to be labeled
  86. consecutively (e.g., 0, 1, 2, …, N-1), while header columns remain unchanged.
  87. For each numeric column i, the output value is computed over a window of size `window_size`
  88. centered at i. If the window would extend beyond the available columns (near the edges),
  89. it is truncated and the operation is performed on the available subset (with at least 2 values).
  90. Parameters
  91. ----------
  92. df : pandas.DataFrame
  93. Input DataFrame containing header columns and numeric columns labeled 0..N-1.
  94. header_cols : list of str
  95. Names of the columns to be preserved as-is in the output.
  96. window_size : int
  97. Size of the sliding window (must be >= 1).
  98. sum : bool, optional
  99. If False (default), computes the mean within each sliding window.
  100. If True, computes the sum instead.
  101. Returns
  102. -------
  103. pandas.DataFrame
  104. DataFrame with the same columns and shape as the input.
  105. Header columns are preserved unchanged, while numeric columns are replaced
  106. by their windowed average or sum (depending on `sum`).
  107. """
  108. numeric_cols = [col for col in df.columns if col not in header_cols]
  109. if sum:
  110. rolled = df[numeric_cols].rolling(window=window_size, axis=1, center=True, min_periods=2).sum()
  111. else:
  112. rolled = df[numeric_cols].rolling(window=window_size, axis=1, center=True, min_periods=2).mean()
  113. # Reconstruct the DataFrame: keep headers, replace numeric columns with rolled values.
  114. result_df = pd.concat([df[header_cols].reset_index(drop=True), rolled.reset_index(drop=True)], axis=1)
  115. # Ensure numeric column names stay as ints (or as they were originally).
  116. result_df.columns = header_cols + [int(c) for c in numeric_cols]
  117. return result_df
  118. # Not used in the final paper
  119. def aggregate_every_3cols(df, header_cols, window_size=3):
  120. """
  121. Aggregates consecutive groups of numeric columns into their rowwise mean, while
  122. preserving the specified header columns. By default, the numeric columns are
  123. processed in blocks of 3, but the block size can be adjusted with `window_size`.
  124. The number of numeric columns must be a multiple of `window_size`. Each new
  125. aggregated column is named according to the range of original columns it summarizes
  126. (e.g., "0_to_2_mean" for columns [0, 1, 2]).
  127. Parameters
  128. ----------
  129. df : pandas.DataFrame
  130. Input DataFrame containing header columns and numeric columns.
  131. header_cols : list of str
  132. Names of the columns to be preserved as-is in the output.
  133. window_size : int, optional
  134. Number of consecutive numeric columns to aggregate (default is 3).
  135. Returns
  136. -------
  137. pandas.DataFrame
  138. A DataFrame containing the header columns and one new column per block of
  139. `window_size` numeric columns, each holding the rowwise mean of that block.
  140. """
  141. # 1) Split off non-numeric headers
  142. numeric_cols = [col for col in df.columns if col not in header_cols]
  143. header = df[header_cols]
  144. nums = df[numeric_cols]
  145. # 2) Check we have a multiple of 3 (optional)
  146. if len(nums.columns) % window_size != 0:
  147. raise ValueError(f"Expected a multiple of {window_size} numeric columns, got {len(nums.columns)}")
  148. # 3) For each block of 3, compute the mean
  149. new_cols = {}
  150. cols = list(nums.columns)
  151. for block_idx in range(0, len(cols), window_size):
  152. trio = cols[block_idx:block_idx + window_size]
  153. # name the new column after the first of the trio (or anything you like)
  154. new_name = f"{trio[0]}_to_{trio[-1]}_mean"
  155. new_cols[new_name] = nums[trio].mean(axis=1)
  156. # 4) Concatenate header + new means
  157. result = pd.concat([header, pd.DataFrame(new_cols, index=df.index)], axis=1)
  158. return result
  159. # region ======================================== Correlation ==========================================================
  160. def perceptual_magnitude_behavior_corr(recs):
  161. """
  162. Computes the correlation between perceptual magnitude (here: % responsive excitatory neurons)
  163. and behavioral outcome (Hit vs Miss) for threshold-level trials using a point-biserial correlation.
  164. This function restricts the analysis to stimulation trials at the session threshold amplitude
  165. in order to avoid bias. It cannot be applied to all trials simultaneously, because amplitude
  166. would need to be included as a covariate.
  167. Parameters
  168. ----------
  169. recs : list
  170. List of recording objects, each expected to provide:
  171. - rec.filename (str) : Unique identifier for the recording.
  172. - rec.genotype (str) : Genotype label.
  173. - rec.session_threshold (float or int) : Threshold amplitude used in the session.
  174. - rec.stim_ampl_filter(stim_ampl="all") : Boolean mask for threshold trials.
  175. - rec.get_perc_resp(pattern=1, n_type="EXC") : Returns excitatory responsivity vector.
  176. - rec.detected_stim : Boolean array of behavioral detection outcomes.
  177. Returns
  178. -------
  179. pandas.DataFrame
  180. A DataFrame where each row corresponds to a recording, with the following columns:
  181. - "ID" : Recording filename.
  182. - "Genotype" : Genotype of the animal.
  183. - "session_threshold" : Threshold amplitude of the session.
  184. - "nb_threshold_trials" : Number of threshold trials included in the correlation.
  185. - "R2" : Coefficient of determination (r²) from the point-biserial correlation.
  186. - "p_val" : Associated p-value.
  187. """
  188. rows = []
  189. for rec in recs:
  190. # Keeping only the stimulation at threshold amplitude to limit bias
  191. threshold_trials_mask = rec.stim_ampl_filter(stim_ampl="all")
  192. # Getting the vector of the parameter to correlate with the behavior
  193. act_exc_vector = rec.get_perc_resp(pattern=1, n_type="EXC")[threshold_trials_mask]
  194. # Getting the vector of behavioral outcome
  195. behavior_vector = rec.detected_stim[threshold_trials_mask]
  196. if len(behavior_vector) > 2:
  197. # Point biserial correlation of the vectors
  198. r, p_val = pointbiserialr(act_exc_vector, behavior_vector)
  199. rows.append({"ID": rec.filename, "Genotype": rec.genotype, "session_threshold": rec.session_threshold,
  200. "nb_threshold_trials": len(behavior_vector), "R2": r**2, "p_val": p_val})
  201. else:
  202. print(f"{rec.filename} {rec.genotype} excluded → only {len(behavior_vector)} threshold trial(s)")
  203. return pd.DataFrame(rows)
  204. def correlate_mean_zscore_behavior_frame(frame_data):
  205. """
  206. Computes, for each recording ID × neuron type × responsivity group, the correlation between
  207. per-frame mean z-scores and behavioral outcome (Hit vs Miss) using point-biserial correlation.
  208. Each row in the output DataFrame corresponds either to the correlation coefficient ("r") or
  209. its associated p-value ("pval") at each frame, for one (ID, Genotype, neuron type, resp) group.
  210. Parameters
  211. ----------
  212. frame_data : pandas.DataFrame
  213. Trial-wise neuronal activity with both header metadata and per-frame z-scores.
  214. Expected columns include:
  215. - "Genotype" : str, genotype label.
  216. - "ID" : str, recording/animal identifier.
  217. - "Threshold" : float or int, session threshold amplitude.
  218. - "Trial" : int, trial index.
  219. - "Amplitude" : float or int, stimulation amplitude.
  220. - "Duration" : float or int, stimulation duration.
  221. - "Behavior" : bool, behavioral outcome (e.g., Hit=True, Miss=False).
  222. - "n_type" : str, neuron type ("EXC" or "INH").
  223. - "resp" : int or bool, responsivity of the neuron in the trial.
  224. - "n_ID" : int, neuron identifier.
  225. - Frame-wise z-scores as numeric columns (0, 1, ..., N).
  226. Returns
  227. -------
  228. pandas.DataFrame
  229. Each row corresponds to one correlation metric for a given (Genotype, ID, n_type, resp) group:
  230. - "Genotype" : genotype label.
  231. - "ID" : recording/animal identifier.
  232. - "n_type" : neuron type.
  233. - "resp" : responsivity group.
  234. - "metric" : "r" (correlation coefficient) or "pval" (associated p-value).
  235. - Frame columns : correlation values (r or pval) at each frame.
  236. Notes
  237. -----
  238. - The function first averages z-scores across neurons (per (ID, Trial, n_type, resp)),
  239. then correlates those per-trial means with the binary behavior vector.
  240. - Only groups with more than one trial are included in the correlation.
  241. - Output rows come in pairs: one with correlation coefficients ("r"), one with p-values ("pval").
  242. """
  243. # Computing the mean zscore per trial
  244. header_columns = ["Genotype", "ID", "Threshold", "Trial", "Amplitude", "Duration", "Behavior", "n_type", "resp", "n_ID"]
  245. mean_accross = ["n_ID"]
  246. grouping_columns = [col for col in header_columns if col not in mean_accross]
  247. data = frame_data.groupby(grouping_columns, as_index=False).mean().drop(columns=mean_accross)
  248. rows = []
  249. for rec_id in data["ID"].unique():
  250. rec_data = data[data["ID"] == rec_id]
  251. genotype = rec_data["Genotype"].values[0]
  252. for neuron_type in rec_data["n_type"].unique():
  253. n_type_data = rec_data[rec_data["n_type"] == neuron_type]
  254. for response in n_type_data["resp"].unique():
  255. resp_data = n_type_data[rec_data["resp"] == response]
  256. row_r = {"Genotype": genotype, "ID": rec_id, "n_type": neuron_type, "resp": response, "metric": "r"}
  257. row_pval = {"Genotype": genotype, "ID": rec_id, "n_type": neuron_type, "resp": response, "metric": "pval"}
  258. y = resp_data["Behavior"].values
  259. if len(resp_data) > 1:
  260. for col in [c for c in data.columns if c not in header_columns]:
  261. x = resp_data[col].values
  262. row_r[col], row_pval[col] = pointbiserialr(x, y)
  263. rows.append(row_r)
  264. rows.append(row_pval)
  265. return pd.DataFrame(rows)
  266. def plot_frame_correlation(corr_data):
  267. """
  268. Plots frame-by-frame correlation results (r and p-values) between neuronal activity
  269. and behavioral outcome, separated by genotype, neuron type, and responsivity.
  270. Parameters
  271. ----------
  272. corr_data : pandas.DataFrame
  273. Correlation results produced by `correlate_mean_zscore_behavior_frame`.
  274. Expected columns include:
  275. - "Genotype" : str, genotype label.
  276. - "ID" : str, recording/animal identifier.
  277. - "n_type" : str, neuron type ("EXC" or "INH").
  278. - "resp" : int or bool, responsivity group.
  279. - "metric" : str, correlation metric ("r" or "pval").
  280. - Frame columns : correlation values (float) for each frame index.
  281. Returns
  282. -------
  283. None
  284. Displays a matplotlib figure with subplots:
  285. - Rows correspond to correlation metrics ("r", "pval").
  286. - Columns correspond to genotypes.
  287. - Each line corresponds to one (ID, n_type, resp) group, with color
  288. indicating neuron type and responsivity.
  289. The plot includes the following visual guides:
  290. - Vertical dashed lines: stimulus onset (frame 30, red) and offset (frame 45, black).
  291. - For p-values, a horizontal green dashed line marks the 0.05 significance threshold.
  292. - Color scheme:
  293. * EXC: light to dark blue for increasing responsivity levels.
  294. * INH: light to dark magenta for increasing responsivity levels.
  295. - Figure title: "Frame by frame correlation of zscore with behavior".
  296. """
  297. color_dict = {"EXC": ["skyblue", "blue", "navy"], "INH": ["pink", "magenta", "darkviolet"]}
  298. fig, ax = plt.subplots(nrows=2, ncols=3, figsize=(18, 12), constrained_layout=True)
  299. for col, genotype in enumerate(corr_data["Genotype"].unique()):
  300. for row, metric in enumerate(corr_data["metric"].unique()):
  301. data = corr_data[(corr_data["Genotype"] == genotype) & (corr_data["metric"] == metric)].drop(columns=["Genotype", "metric"])
  302. header_columns = ["ID", "n_type", "resp"]
  303. mean_accross = ["ID"]
  304. grouping_columns = [col for col in header_columns if col not in mean_accross]
  305. data = data.groupby(grouping_columns, as_index=False).mean().drop(columns=mean_accross)
  306. # Plotting
  307. ax[row, col].set_title(f"{metric} for {genotype}")
  308. for i, curve_row in data.iterrows():
  309. y = curve_row.drop(labels=["n_type", "resp"]).values.tolist()
  310. x = np.arange(len(y))
  311. ax[row, col].plot(x, y, color=color_dict[curve_row["n_type"]][int(curve_row["resp"])], lw=1)
  312. ax[row, col].axvline(x=30, ls="--", lw=1, color="red")
  313. ax[row, col].axvline(x=45, ls="--", lw=1, color="black")
  314. if metric == "pval":
  315. ax[row, col].axhline(y=0.05, ls="--", lw=1, color="green")
  316. fig.suptitle("Frame by frame correlation of zscore with behavior")
  317. fig.canvas.manager.set_window_title("Frame_corr_zscore_behavior")
  318. plt.show()
  319. # endregion ============================================================================================================
  320. # region ======================================== Modelling ============================================================
  321. # Not used in the final paper
  322. def glmm_behavior(data):
  323. """
  324. Fits a generalized linear mixed model (GLMM) to explain the behavioral outcome
  325. from neuronal predictors and stimulation parameters.
  326. Parameters
  327. ----------
  328. data : pandas.DataFrame
  329. Long-format dataset containing:
  330. - "behavior" : binary outcome (trial detected = 1, not detected = 0).
  331. - "Genotype" : categorical factor with levels ["WT", "KO", "KO-Hypo"].
  332. - "ID" : subject/recording identifier (random effect).
  333. - "amplitude" : stimulus amplitude.
  334. - "act_EXC_perc", "inh_EXC_perc", "act_INH_perc", "inh_INH_perc" :
  335. trial-level neuronal activity measures used as predictors.
  336. Returns
  337. -------
  338. result : statsmodels.genmod.generalized_linear_model.BinomialBayesMixedGLMResults
  339. Fitted GLMM object. A summary of the model is printed to console.
  340. The model is binomial with a logit link, and uses the following specification:
  341. ``behavior ~ amplitude*Genotype
  342. + amplitude:act_EXC_perc*Genotype
  343. + amplitude:inh_EXC_perc*Genotype
  344. + amplitude:act_INH_perc*Genotype
  345. + amplitude:inh_INH_perc*Genotype``
  346. Random effects are modeled by subject ID, with one variance component per subject.
  347. Estimation is done using variational Bayes (`fit_vb`). The function contains
  348. commented-out alternatives for fitting a GEE restricted to WT animals or
  349. including additional predictors (e.g. amplitude, delay).
  350. """
  351. data["Genotype"] = pd.Categorical(data["Genotype"], categories=["WT", "KO", "KO-Hypo"], ordered=True)
  352. # data["behavior"] = pd.Categorical(data["behavior"], categories=["False", "True"], ordered=True)
  353. # data["behavior"] = data["behavior"].astype(int)
  354. # === === Fitting the model === ===
  355. # --- GEE ---
  356. formula = ("behavior ~ amplitude*Genotype + amplitude:act_EXC_perc*Genotype + amplitude:inh_EXC_perc*Genotype + "
  357. "amplitude:act_INH_perc*Genotype + amplitude:inh_INH_perc*Genotype")# + "
  358. # # "act_EXC_amp + inh_EXC_amp + act_INH_amp + inh_INH_amp + "
  359. # # "act_EXC_delay + inh_EXC_delay + act_INH_delay + inh_INH_delay + "
  360. # # "Genotype + amplitude:Genotype")
  361. # gee_model = smf.gee(formula, groups="ID", data=data[data["Genotype"] == "WT"], family=sm.families.Binomial())
  362. # gee_result = gee_model.fit()
  363. # print(gee_result.summary())
  364. # --- GLMM ---
  365. vc_formulas = {"ID": "0 + C(ID)"}
  366. model = sm.genmod.BinomialBayesMixedGLM.from_formula(formula, vc_formulas, data=data)#, family=sm.families.Binomial())
  367. result = model.fit_vb()
  368. print(result.summary())
  369. # Not used in the final paper
  370. def frame_model_n_type_avg(frame_data):
  371. """
  372. Trains a logistic regression model for each frame and each animal to classify
  373. behavioral outcome (hit vs. miss) based on averaged neuronal responses.
  374. Cross-validation is used to assess classification accuracy, with undersampling
  375. performed to correct class imbalance.
  376. For each frame, a pivoted dataset is constructed with columns corresponding to
  377. (n_type, resp) neuron categories. Trials with amplitude = 0 and inhibited INH
  378. neurons are excluded. Missing values are imputed by the per-animal/per-amplitude
  379. mean. Logistic regression with L2 regularization is then trained using
  380. undersampled training data, and evaluated on a held-out test set.
  381. Parameters
  382. ----------
  383. frame_data : pandas.DataFrame
  384. Long-format trial-level dataset containing:
  385. - "Genotype" : animal genotype.
  386. - "ID" : animal/session identifier.
  387. - "Threshold" : stimulation threshold for that session.
  388. - "Trial" : trial index.
  389. - "Amplitude" : stimulation amplitude.
  390. - "Duration" : stimulation duration.
  391. - "Behavior" : binary outcome (hit/miss).
  392. - "n_type" : neuron type (EXC or INH).
  393. - "resp" : response sign (-1, 0, 1).
  394. - "n_ID" : neuron identifier.
  395. - plus z-score values across frames.
  396. Returns
  397. -------
  398. results_df : pandas.DataFrame
  399. A dataframe where each row corresponds to one animal and one frame.
  400. Includes:
  401. - "Genotype" : animal genotype.
  402. - "ID" : animal/session identifier.
  403. - "Threshold" : stimulation threshold.
  404. - "Frame" : frame index.
  405. - "TPR" : true positive rate (sensitivity/recall).
  406. - "FPR" : false positive rate.
  407. - "Accuracy" : classification accuracy.
  408. Logistic regression uses stratified train/test splits, undersampling
  409. within the training set, and evaluation on the held-out test set.
  410. The confusion matrix is used to compute the reported metrics.
  411. """
  412. # Grouping the different neurons
  413. header_columns = ["Genotype", "ID", "Threshold", "Trial", "Amplitude", "Duration", "Behavior", "n_type", "resp",
  414. "n_ID"]
  415. mean_accross = ["n_ID"]
  416. grouping_columns = [col for col in header_columns if col not in mean_accross]
  417. data = frame_data.groupby(grouping_columns, as_index=False).mean().drop(columns=mean_accross)
  418. # Creating a pivot DataFrame to obtain a dataframe per frame, each column being a neuron type/resp combinaison
  419. index_columns = [col for col in grouping_columns if col not in ["n_type", "resp"]]
  420. numeric_columns = [c for c in data.columns if c not in header_columns]
  421. rows = []
  422. for frame in numeric_columns:
  423. print(f"Frame n°{frame}")
  424. frame_data = data.pivot(index=index_columns, columns=["n_type", "resp"], values=frame).reset_index()
  425. # Dropping no go trials and the inhibited INH neurons because they are too few and induce NaN values
  426. frame_data = frame_data.drop(columns=("INH", -1))
  427. frame_data = frame_data[frame_data["Amplitude"] != 0]
  428. # Imputing the NaN by the mean value per animal per amplitude
  429. for col in [("EXC", 0), ("EXC", 1), ("EXC", -1), ("INH", 0), ("INH", 1)]:
  430. frame_data[col] = frame_data.groupby(["Genotype", "ID", "Threshold", "Amplitude"], as_index=False)[[col]].transform(lambda x: x.fillna(x.mean()))
  431. # Training a model for each recording and storing the evaluation metrics in a new DataFrame
  432. for rec_id in frame_data["ID"].unique():
  433. filtered_data = frame_data[frame_data["ID"] == rec_id]
  434. X = filtered_data.drop(columns=index_columns)
  435. y = filtered_data["Behavior"]
  436. # Splitting the data into training and test sets (using stratification to preserve class distribution)
  437. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42, stratify=y)
  438. # 1) Use CV to find best C value for L2 regularization of LR, undersample each fold
  439. # pipeline = Pipeline(steps=[('under', RandomUnderSampler(random_state=42)),
  440. # ('clf', LogisticRegression(solver='lbfgs', max_iter=1000))])
  441. # 2) Train the LR with the best parameters on the global undersampled X_train
  442. # 3) Assess model performance on test set
  443. # Create a pipeline that first undersamples then fits a logistic regression model
  444. undersampler = RandomUnderSampler(random_state=42)
  445. X_train_res, y_train_res = undersampler.fit_resample(X_train, y_train)
  446. lr = LogisticRegression(solver='lbfgs', C=1, max_iter=5000)
  447. # Set up a grid of hyperparameters to tune
  448. # param_grid = {"clf__C": [0.00001, 0.0001, 0.001, 0.01, 0.1, 1, 10]}
  449. # Use cross-validation (here, 5-fold) to search for the best hyperparameters
  450. # grid_search = GridSearchCV(pipeline, param_grid, cv=4, scoring='accuracy')
  451. # grid_search.fit(X_train, y_train)
  452. # Evaluate on the test set
  453. # y_pred = grid_search.predict(X_test)
  454. lr.fit(X_train_res, y_train_res)
  455. y_pred = lr.predict(X_test)
  456. cm = confusion_matrix(y_test, y_pred)
  457. # Calculate true positive and true negative rates
  458. tn, fp, fn, tp = cm.ravel()
  459. tpr = tp / (tp + fn) # Sensitivity / Recall
  460. fpr = fp / (tn + fp)
  461. accuracy = (tp + tn) / (tp + tn + fp + fn)
  462. # Optionally, print a full classification report
  463. # print(classification_report(y_test, y_pred))
  464. row = {"Genotype": filtered_data["Genotype"].values[0], "ID": filtered_data["ID"].values[0],
  465. "Threshold": filtered_data["Threshold"].values[0], "Frame": frame, "TPR": tpr, "FPR": fpr, "Accuracy": accuracy}
  466. rows.append(row)
  467. return pd.DataFrame(rows)
  468. def frame_model(frame_data, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling",
  469. sliding_window=None, window_sum=False, window=None):
  470. """
  471. Trains logistic regression models to classify behavioral outcome (hit vs. miss)
  472. based on neuronal responses across frames. For each frame and each animal,
  473. a dataset of neuronal activity is pivoted so that each neuron forms one predictor.
  474. Logistic regression is then trained and evaluated either with nested double
  475. cross-validation or a train/test split, depending on the `db_cv` flag.
  476. To handle class imbalance, the user can choose resampling, sample weighting,
  477. or no balancing method. Optionally, neuronal activity can be aggregated
  478. over time using a sliding window or a fixed-size window. For each animal and frame,
  479. the model reports classification metrics including accuracy, sensitivity (TPR),
  480. and false positive rate (FPR), along with shuffled controls.
  481. Parameters
  482. ----------
  483. frame_data : pandas.DataFrame
  484. Trial-level dataset containing:
  485. - "Genotype" : animal genotype.
  486. - "ID" : animal/session identifier.
  487. - "Trial" : trial index.
  488. - "Amplitude" : stimulation amplitude.
  489. - "Duration" : stimulation duration.
  490. - "Behavior" : binary outcome (hit/miss).
  491. - "n_type" : neuron type (EXC or INH).
  492. - "resp" : response sign (-1, 0, 1).
  493. - "n_ID" : neuron identifier.
  494. - plus z-score values across frames.
  495. neuron_type : list of str, default=["EXC", "INH"]
  496. Neuron types to include in the analysis.
  497. resp_type : list of int, default=[0, 1, -1]
  498. Response categories to include (inactive, activated, inhibited).
  499. db_cv : bool, default=True
  500. If True, performs nested double cross-validation to tune hyperparameters
  501. and evaluate generalization. If False, a single train/test split is used.
  502. balancing_method : {"resampling", "weights", None}, default="resampling"
  503. Strategy to address class imbalance:
  504. - "resampling" : undersampling with RandomUnderSampler.
  505. - "weights" : class_weight="balanced" in logistic regression.
  506. - None : no explicit balancing.
  507. sliding_window : int or None, default=None
  508. Size of the sliding window (in frames) for temporal averaging of activity.
  509. Mutually exclusive with `window`.
  510. window_sum : bool, default=False
  511. If True and `sliding_window` is provided, sums activity within the window
  512. instead of averaging.
  513. window : int or None, default=None
  514. Size of non-overlapping windows (in frames) for aggregation.
  515. Mutually exclusive with `sliding_window`.
  516. Returns
  517. -------
  518. frame_model_df : pandas.DataFrame
  519. Dataframe where each row corresponds to one animal and one frame,
  520. containing:
  521. - "Genotype", "ID", "Frame" : identifiers.
  522. - "TPR" : true positive rate (sensitivity/recall).
  523. - "FPR" : false positive rate.
  524. - "Accuracy" : classification accuracy.
  525. - "TPR_shuffle", "FPR_shuffle", "Accuracy_shuffle" :
  526. performance metrics after label shuffling (baseline control).
  527. frame_mean_sem : matplotlib.Figure or tuple
  528. Summary plot of classification accuracy over frames, averaged across animals
  529. with SEM. Title encodes selected neuron/response types and whether double CV
  530. was used.
  531. """
  532. data = frame_data.copy()
  533. assert (sliding_window is None or window is None), "Please choose between each frame, sliding window, and window"
  534. header_columns = ["Genotype", "ID", "Threshold", "Trial", "Amplitude", "Duration", "Behavior", "n_type", "resp", "n_ID"]
  535. if sliding_window is not None:
  536. data = sliding_window_average(frame_data, header_columns, sliding_window, sum=window_sum)
  537. if window is not None:
  538. data = aggregate_every_3cols(frame_data, header_columns, window_size=window)
  539. # Filtering the neuron types and activity
  540. data = data[data["n_type"].isin(neuron_type)]
  541. data = data[data["resp"].isin(resp_type)]
  542. data = data.drop(columns=["Threshold", "Amplitude", "Duration", "resp"])
  543. index_columns = ["Genotype", "ID", "Trial", "Behavior", "n_type", "n_ID"]
  544. numeric_columns = [c for c in data.columns if c not in header_columns]
  545. rows = []
  546. # Creating a pivot DataFrame to obtain a dataframe per frame, each column being a neuron
  547. for frame in tqdm(numeric_columns):
  548. frame_data = frame_data[frame_data["Amplitude"] != 0]
  549. # Training a model for each recording and storing the evaluation metrics in a new DataFrame
  550. for rec_id in data["ID"].unique():
  551. rec_data = data[data["ID"] == rec_id].copy()
  552. rec_data["Neuron"] = rec_data["n_ID"].astype(str) + "_" + rec_data["n_type"].astype(str)
  553. final_data = rec_data.pivot(index=["Trial", "Behavior"], columns="Neuron", values=frame).reset_index()
  554. y = final_data["Behavior"]
  555. X = final_data.drop(columns=["Trial", "Behavior"])
  556. if db_cv:
  557. metrics = double_cv(np.array(X), np.array(y), cv_out_fold=4, cv_in_fold=4,
  558. param_grid={"C": [0.0001, 0.001, 0.01, 0.1, 1], "penalty": ["l2"]},
  559. scoring_metric="Accuracy", resampler=RandomUnderSampler(random_state=42),
  560. random_state=42, get_df=False)
  561. else:
  562. # Splitting the data into training and test sets (using stratification to preserve class distribution)
  563. X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y)
  564. if balancing_method == "resampling":
  565. undersampler = RandomUnderSampler(random_state=42)
  566. X_train_res, y_train_res = undersampler.fit_resample(X_train, y_train)
  567. lr = LogisticRegression(solver='lbfgs', C=1, max_iter=5000)
  568. lr.fit(X_train_res, y_train_res)
  569. elif balancing_method == "weights":
  570. # Uses the weight parameter of the LR to balance the weight of samples rather than resampling
  571. lr = LogisticRegression(solver='lbfgs', class_weight="balanced", C=1, max_iter=5000)
  572. lr.fit(X_train, y_train)
  573. elif balancing_method == None:
  574. lr = LogisticRegression(solver='lbfgs', C=1, max_iter=5000)
  575. lr.fit(X_train, y_train)
  576. y_pred = lr.predict(X_test)
  577. metrics = get_metrics(y_test, y_pred)
  578. row = {"Genotype": rec_data["Genotype"].values[0], "ID": rec_data["ID"].values[0],
  579. "Frame": frame, "TPR": metrics["TPR"], "FPR": metrics["FPR"], "Accuracy": metrics["Accuracy"],
  580. "TPR_shuffle": metrics["TPR_shuffle"], "FPR_shuffle": metrics["FPR_shuffle"], "Accuracy_shuffle": metrics["Accuracy_shuffle"]}
  581. # "p_hit": metrics["p_hit"], "p_miss": metrics["p_miss"]}
  582. rows.append(row)
  583. frame_model_df = pd.DataFrame(rows)
  584. frame_mean_sem = plot_hit_miss_classif(frame_model_df, title_precision=f"{neuron_type}{resp_type} - Double CV=={db_cv}", timescale_division_factor=(1 if window is None else window))
  585. # frame_comp = plot_hit_miss_classif_comp(frame_model_df, gp1="WT", gp2="KO-Hypo", title_precision=f"{neuron_type}{resp_type} - db_cv={db_cv}")
  586. return frame_model_df, frame_mean_sem #, frame_comp
  587. def get_metrics(y_test, y_pred):
  588. """
  589. Compute basic binary classification metrics from test labels and predictions.
  590. The confusion matrix is constructed with labels ordered as [False, True],
  591. so that rows correspond to the true class and columns to the predicted class.
  592. From this, true positives (TP), true negatives (TN), false positives (FP),
  593. and false negatives (FN) are extracted, and the following metrics are calculated:
  594. - TPR (True Positive Rate, also called Sensitivity or Recall): TP / (TP + FN).
  595. - FPR (False Positive Rate): FP / (TN + FP).
  596. - Accuracy: (TP + TN) / (TP + TN + FP + FN).
  597. Parameters
  598. ----------
  599. y_test : array-like of shape (n_samples,)
  600. Ground truth binary labels (must contain values interpretable as True/False).
  601. y_pred : array-like of shape (n_samples,)
  602. Predicted binary labels.
  603. Returns
  604. -------
  605. metrics : dict
  606. Dictionary containing:
  607. - "TPR" : True Positive Rate (recall).
  608. - "FPR" : False Positive Rate.
  609. - "Accuracy" : Overall classification accuracy.
  610. """
  611. metrics = {}
  612. cm = confusion_matrix(y_test, y_pred, labels=[False, True])
  613. TP = cm[1, 1]
  614. TN = cm[0, 0]
  615. FP = cm[0, 1]
  616. FN = cm[1, 0]
  617. metrics["TPR"] = TP / (TP + FN) # Sensitivity / Recall
  618. metrics["FPR"] = FP / (TN + FP)
  619. metrics["Accuracy"] = (TP + TN) / (TP + TN + FP + FN)
  620. return metrics
  621. def double_cv(X, y, cv_out_fold=5, cv_in_fold=5, param_grid={"C": [0.0001, 0.001, 0.01, 0.1, 1], "penalty": ["l2"]},
  622. scoring_metric="Accuracy", resampler=RandomUnderSampler(random_state=42), random_state=42, get_df=False):
  623. """
  624. Perform nested (double) cross-validation with logistic regression for hyperparameter tuning,
  625. class rebalancing, and performance benchmarking against shuffled labels.
  626. The outer cross-validation loop (cv_out_fold splits) estimates generalization performance,
  627. while the inner loop (cv_in_fold splits) selects the best hyperparameters from `param_grid`.
  628. A resampling strategy (default: random undersampling) can be applied to address class imbalance.
  629. After model fitting, both performance on true labels and a shuffled-label baseline are reported
  630. to assess whether classification accuracy is above chance.
  631. Parameters
  632. ----------
  633. X : ndarray of shape (n_samples, n_features)
  634. Input feature matrix.
  635. y : ndarray of shape (n_samples,)
  636. Binary labels corresponding to X.
  637. cv_out_fold : int, default=5
  638. Number of outer folds for performance evaluation.
  639. cv_in_fold : int, default=5
  640. Number of inner folds for hyperparameter tuning.
  641. param_grid : dict, default={"C": [0.0001, 0.001, 0.01, 0.1, 1], "penalty": ["l2"]}
  642. Grid of logistic regression hyperparameters to explore.
  643. scoring_metric : {"Accuracy", "TPR", "FPR"}, default="Accuracy"
  644. Metric used to select the best hyperparameters during the inner CV.
  645. resampler : imbalanced-learn resampler or None, default=RandomUnderSampler(random_state=42)
  646. Strategy to balance classes. If None, no resampling is applied.
  647. random_state : int, default=42
  648. Random seed for reproducibility in CV splitting and resampling.
  649. get_df : bool, default=False
  650. If True, return the full per-fold DataFrame. If False, return the mean metrics.
  651. Returns
  652. -------
  653. results : dict or pandas.DataFrame
  654. - If get_df=False: dictionary with mean performance metrics across outer folds, e.g.:
  655. {"TPR": ..., "FPR": ..., "Accuracy": ..., "TPR_shuffle": ..., "FPR_shuffle": ..., "Accuracy_shuffle": ...}.
  656. - If get_df=True: DataFrame containing results for each outer fold, including best parameters and both
  657. true-label and shuffled-label performance.
  658. Notes
  659. -----
  660. - True performance is compared against a shuffled-label baseline, highlighting whether the classifier
  661. performs better than chance.
  662. - Metrics are computed using `get_metrics`, which returns TPR (recall), FPR, and Accuracy.
  663. - Logistic regression models are trained with `max_iter=5000` to ensure convergence.
  664. """
  665. y_true = []
  666. y_pred = []
  667. cv_out = StratifiedKFold(n_splits=cv_out_fold, random_state=random_state, shuffle=True)
  668. rows = []
  669. # Splitting the data into training and validation sets
  670. for fold_out, (train_index, val_index) in enumerate(cv_out.split(X, y)):
  671. row = {"Fold": fold_out}
  672. X_train, X_val, y_train, y_val = X[train_index], X[val_index], y[train_index], y[val_index]
  673. # Splitting the train data into tuning and tuning assessment group
  674. cv_in = StratifiedKFold(n_splits=cv_in_fold, random_state=random_state, shuffle=True)
  675. inner_scores = {}
  676. param_names = list(param_grid.keys())
  677. param_combinations = list(product(*[param_grid[p] for p in param_names]))
  678. for params in param_combinations:
  679. params_dict = dict(zip(param_names, params))
  680. # Performing CV to find the best hyperparameters
  681. fold_scores = []
  682. for fold_in, (tuning_index, test_index) in enumerate(cv_in.split(X_train, y_train)):
  683. X_tuning, X_test, y_tuning, y_test = X_train[tuning_index], X_train[test_index], y_train[tuning_index], y_train[test_index]
  684. # Tuning hyper-parameters on the tuning data set
  685. model = LogisticRegression(**params_dict, max_iter=5000, random_state=random_state)
  686. model.fit(X_tuning, y_tuning)
  687. # Predicting the test set and storing metrics
  688. y_pred_in = model.predict(X_test)
  689. metrics = get_metrics(y_test, y_pred_in)
  690. fold_scores.append(metrics[scoring_metric])
  691. inner_scores[tuple(params)] = np.mean(fold_scores)
  692. # Choosing the best parameters from the inner loop
  693. best_params_tuple = max(inner_scores, key=inner_scores.get)
  694. best_params = dict(zip(param_names, best_params_tuple))
  695. row["Best_param"] = best_params
  696. # Resampling the training set and training the final model with this set
  697. if resampler is not None:
  698. X_train_res, y_train_res = resampler.fit_resample(X_train, y_train)
  699. else:
  700. X_train_res, y_train_res = X_train, y_train
  701. final_model = LogisticRegression(**best_params, max_iter=5000, random_state=random_state)
  702. final_model.fit(X_train_res, y_train_res)
  703. # Assessing the final model's performance
  704. y_val_pred = final_model.predict(X_val)
  705. outer_metrics = get_metrics(y_val, y_val_pred)
  706. row.update(outer_metrics)
  707. rows.append(row)
  708. # 1) Training the model on shuffled labels to compare its performance
  709. y_train_res_shuffled = y_train_res.copy()
  710. random.shuffle(y_train_res_shuffled)
  711. final_model.fit(X_train_res, y_train_res_shuffled)
  712. # Assessing the final model's performance
  713. y_val_pred_shuffle = final_model.predict(X_val)
  714. outer_metrics_shuffle = get_metrics(y_val, y_val_pred_shuffle)
  715. outer_metrics_shuffle = {f"{k}_shuffle": v for k, v in outer_metrics_shuffle.items()}
  716. row.update(outer_metrics_shuffle)
  717. rows.append(row)
  718. # 2) Rebuilding true and predicted label vectors to compare the obtained results to chance with binomial test
  719. # y_true.extend(y_val)
  720. # y_pred.extend(y_val_pred)
  721. # Computing the chance that the obtained classification is better than chance levels
  722. # nb_TP = int(np.sum((y_true == 1) & (y_pred == 1)))
  723. # nb_hit = int(np.sum(y_true))
  724. # p_hit = np.nan if nb_hit==0 else binomtest(nb_TP, nb_hit, p=0.5, alternative="greater").pvalue
  725. # print(f"Nb TP: {nb_TP}, Nb hit: {nb_hit}, P: {p_hit}")
  726. # nb_FP = int(np.sum((y_true == 0) & (y_pred == 1)))
  727. # nb_miss = int(len(y_true) - nb_hit)
  728. # p_miss = np.nan if nb_miss==0 else binomtest(nb_FP, nb_miss, p=0.5, alternative="less").pvalue
  729. # print(f"Nb FP: {nb_FP}, Nb miss: {nb_miss}, P: {p_miss}")
  730. results_df = pd.DataFrame(rows)
  731. results_metrics = results_df.mean(numeric_only=True).to_dict()
  732. # results_metrics.update({"p_hit": p_hit, "p_miss": p_miss})
  733. return results_df if get_df else results_metrics
  734. def plot_hit_miss_classif(frame_model_df, title_precision="", shuffle=False, shuffle_frame_significance=False,
  735. timescale_division_factor=1):
  736. """
  737. Plot hit versus miss classification curves from a DataFrame, where each row contains information about TPR and FPR
  738. for one frame and one animal. The function averages these values across animals of the same genotype and plots
  739. mean ± SEM curves, optionally comparing to shuffled-label baselines.
  740. The plot includes:
  741. - True Positive Rate (TPR, "hit accuracy") and False Positive Rate (FPR, "miss accuracy") across time.
  742. - Shuffled-label curves (dotted, semi-transparent) if `shuffle=True`.
  743. - Statistical significance markers (horizontal bars) on frames where real vs. shuffled curves differ, if
  744. `shuffle_frame_significance=True`.
  745. - Vertical dashed lines marking stimulus onset (red, at 30/timescale_division_factor) and offset (black, at
  746. 45/timescale_division_factor).
  747. - A horizontal dashed line at 0.5, representing chance level.
  748. - x-axis ticks aligned to -1, -0.5, 0, 0.5, 1 seconds relative to stimulus timing.
  749. Parameters
  750. ----------
  751. frame_model_df : pd.DataFrame
  752. Input DataFrame containing columns:
  753. - "Genotype": categorical group label for each animal (e.g., "WT", "KO").
  754. - "ID": animal identifier.
  755. - "Frame": time frame index.
  756. - "TPR", "FPR": classification metrics for each animal and frame.
  757. Optionally includes "TPR_shuffle" and "FPR_shuffle" if `shuffle=True`.
  758. title_precision : str, optional
  759. Extra string appended to the figure title and window title (default = "").
  760. shuffle : bool, optional
  761. If True, also plot performance curves from shuffled labels for comparison (default = False).
  762. shuffle_frame_significance : bool, optional
  763. If True and `shuffle=True`, performs Wilcoxon signed-rank tests across animals for each frame to
  764. mark frames where shuffled vs. real data differ significantly (default = False).
  765. timescale_division_factor : int, optional
  766. Factor dividing the frame index to adjust x-axis scale (useful for plotting at different time resolutions,
  767. default = 1).
  768. Returns
  769. -------
  770. pd.DataFrame
  771. Subset of the input DataFrame corresponding to the last processed genotype, with "Genotype" and "ID" columns
  772. dropped.
  773. """
  774. draw_style = "default" if timescale_division_factor == 1 else "steps-post"
  775. fill_style = None if timescale_division_factor == 1 else "post"
  776. if "WT-BMS" in frame_model_df["Genotype"].unique():
  777. fig, ax = plt.subplots(nrows=1, ncols=4, figsize=(28, 8), constrained_layout=True)
  778. else:
  779. fig, ax = plt.subplots(nrows=1, ncols=3, figsize=(21, 8), constrained_layout=True)
  780. for i, genotype in enumerate(frame_model_df["Genotype"].unique()):
  781. data = frame_model_df[frame_model_df["Genotype"] == genotype].drop(columns=["Genotype", "ID"])
  782. # Curves definition
  783. tpr_mean = data.groupby("Frame")["TPR"].mean().values
  784. tpr_sem = data.groupby("Frame")["TPR"].sem().values
  785. fpr_mean = data.groupby("Frame")["FPR"].mean().values
  786. fpr_sem = data.groupby("Frame")["FPR"].sem().values
  787. # Curves plotting
  788. x = np.arange(len(tpr_mean))
  789. ax[i].plot(x, tpr_mean, label="Hit accuracy", color=sty.color_dict[genotype][0], lw=2, drawstyle=draw_style)
  790. ax[i].fill_between(x, tpr_mean - tpr_sem, tpr_mean + tpr_sem, color=sty.color_dict[genotype][0], alpha=0.3, step=fill_style)
  791. ax[i].plot(x, fpr_mean, label="Miss accuracy", color=sty.color_dict[genotype][1], lw=2, drawstyle=draw_style)
  792. ax[i].fill_between(x, fpr_mean - fpr_sem, fpr_mean + fpr_sem, color=sty.color_dict[genotype][1], alpha=0.3, step=fill_style)
  793. if shuffle:
  794. # Shuffle curves definition
  795. tpr_mean_shuffle = data.groupby("Frame")["TPR_shuffle"].mean().values
  796. tpr_sem_shuffle = data.groupby("Frame")["TPR_shuffle"].sem().values
  797. fpr_mean_shuffle = data.groupby("Frame")["FPR_shuffle"].mean().values
  798. fpr_sem_shuffle = data.groupby("Frame")["FPR_shuffle"].sem().values
  799. # Shuffle curves plotting
  800. x = np.arange(len(tpr_mean))
  801. ax[i].plot(x, tpr_mean_shuffle, label="Hit accuracy", color=sty.color_dict[genotype][0], lw=1, ls="dotted", alpha=0.25, drawstyle=draw_style)
  802. ax[i].fill_between(x, tpr_mean_shuffle - tpr_sem_shuffle, tpr_mean_shuffle + tpr_sem_shuffle, color=sty.color_dict[genotype][0], alpha=0.1, step=fill_style)
  803. ax[i].plot(x, fpr_mean_shuffle, label="Miss accuracy", color=sty.color_dict[genotype][1], lw=1, ls="dotted", alpha=0.25, drawstyle=draw_style)
  804. ax[i].fill_between(x, fpr_mean_shuffle - fpr_sem_shuffle, fpr_mean_shuffle + fpr_sem_shuffle, color=sty.color_dict[genotype][1], alpha=0.1, step=fill_style)
  805. if shuffle_frame_significance:
  806. # Computing the statistical difference with shuffle
  807. pvals = {}
  808. for frame in data.Frame.unique():
  809. real_tpr = data[data.Frame == frame]["TPR"]
  810. shuffle_tpr = data[data.Frame == frame]["TPR_shuffle"]
  811. real_fpr = data[data.Frame == frame]["FPR"]
  812. shuffle_fpr = data[data.Frame == frame]["FPR_shuffle"]
  813. # paired test across animals
  814. # drop any animals missing one of the two
  815. idx = real_tpr.index.intersection(shuffle_tpr.index)
  816. if len(idx) >= 3:
  817. stat, p_tpr = wilcoxon(real_tpr.loc[idx], shuffle_tpr.loc[idx], alternative="greater")
  818. stat, p_fpr = wilcoxon(real_fpr.loc[idx], shuffle_fpr.loc[idx], alternative="less")
  819. else:
  820. p_tpr = np.nan
  821. p_fpr = np.nan
  822. pvals[frame] = [p_tpr, p_fpr]
  823. # Drawing a bar above significantly different frames
  824. for f, p_list in pvals.items():
  825. p_tpr = p_list[0]
  826. p_fpr = p_list[1]
  827. if p_tpr < 0.05:
  828. # draw a tiny horizontal line spanning the width of one frame
  829. ax[i].hlines(0.95, f - 0.4, f + 0.4, color=sty.color_dict[genotype][0], linewidth=2)
  830. if p_fpr < 0.05:
  831. # draw a tiny horizontal line spanning the width of one frame
  832. ax[i].hlines(0.05, f - 0.4, f + 0.4, color=sty.color_dict[genotype][1], linewidth=2)
  833. # Delimitation of the stimulus period and chance level
  834. ax[i].axvline(x=30/timescale_division_factor, ls="--", lw=1, color="red")
  835. ax[i].axvline(x=45/timescale_division_factor, ls="--", lw=1, color="black")
  836. ax[i].axhline(y=0.5, ls="--", lw=1, color="gray")
  837. # Title and axes formatting
  838. ax[i].set_title(genotype, color=sty.color_dict[genotype][0], fontsize=20)
  839. ax[i].set_ylim(0, 1)
  840. ax[i].set_xlabel("Time (s)")
  841. ax[i].set_xticks(np.divide([0, 15, 30, 45, 60], timescale_division_factor))
  842. ax[i].set_xticklabels([-1, -0.5, 0, 0.5, 1])
  843. fig.suptitle(f"Hit versus Miss classification graph using the mean ΔF/F\n{title_precision}", fontsize=20)
  844. fig.canvas.manager.set_window_title(f"Hit_Miss_classif_{title_precision}")
  845. plt.show()
  846. return data
  847. def plot_hit_miss_classif_comp(frame_model_df, gp1="WT", gp2="KO-Hypo", title_precision=""):
  848. """
  849. Compare hit accuracy (TPR) and miss error (FPR) between two genotypes across stimulus-related periods,
  850. and evaluate real vs. shuffled performance within each genotype. This function generates two figures:
  851. 1. **Genotype comparison figure (2x4 subplots)**
  852. - Top row: boxplots comparing hit accuracy (TPR) between `gp1` and `gp2` across four time windows.
  853. - Bottom row: boxplots comparing miss error (FPR) between `gp1` and `gp2`.
  854. - Colors follow `sty.color_dict`, different per genotype.
  855. - Significance markers can be displayed depending on the boxplot utility function (`ppt.boxplot`).
  856. 2. **Shuffle comparison figure (4x4 subplots)**
  857. - For each of the four periods, plots real vs. shuffled TPR and FPR separately for `gp1` (rows 0–1)
  858. and `gp2` (rows 2–3).
  859. - Uses paired comparisons (real vs. shuffled for the same animals).
  860. - Real data colored by genotype, shuffled data in gray.
  861. - Significance markers are drawn if differences are detected.
  862. The time windows analyzed are:
  863. - `"stim"`: full stimulus window (frames 30–45).
  864. - `"start_stim (250ms)"`: early stimulus subwindow (frames 30–37).
  865. - `"end_stim (250ms)"`: late stimulus subwindow (frames 37–45).
  866. - `"pre_stim (200ms)"`: baseline window before stimulus onset (frames 24–30).
  867. Parameters
  868. ----------
  869. frame_model_df : pd.DataFrame
  870. Input DataFrame containing columns:
  871. - "Genotype": categorical labels for each animal (e.g., "WT", "KO-Hypo").
  872. - "ID": animal identifier.
  873. - "Frame": time frame index.
  874. - "TPR", "FPR": classification metrics per frame.
  875. - "TPR_shuffle", "FPR_shuffle": shuffled baseline metrics for comparison.
  876. gp1 : str, optional
  877. Name of the first genotype group (default = "WT").
  878. gp2 : str, optional
  879. Name of the second genotype group (default = "KO-Hypo").
  880. title_precision : str, optional
  881. Extra string appended to figure and window titles (default = "").
  882. Returns
  883. -------
  884. dict of pd.DataFrame
  885. Dictionary mapping each analyzed period ("stim", "start_stim (250ms)", "end_stim (250ms)",
  886. "pre_stim (200ms)") to the aggregated DataFrame containing mean values per animal and genotype.
  887. """
  888. period_dict = {"stim": [30, 45], "start_stim (250ms)": [30, 37], "end_stim (250ms)": [37, 45], "pre_stim (200ms)": [24, 30]}
  889. fig, ax = plt.subplots(nrows=2, ncols=4, figsize=(24, 16), constrained_layout=True)
  890. fig_shuf, ax_shuf = plt.subplots(nrows=4, ncols=4, figsize=(24, 32), constrained_layout=True)
  891. data_dict = {}
  892. for col, period in enumerate(period_dict.keys()):
  893. start, end = period_dict[period]
  894. data = frame_model_df[frame_model_df["Frame"].isin(range(start, end))].groupby(["Genotype", "ID"], as_index=False).mean().drop(columns="Frame")
  895. # Plotting the comparison of accuracy between genotypes
  896. ppt.boxplot(ax[0, col], data[data["Genotype"] == gp1]["TPR"], data[data["Genotype"] == gp2]["TPR"], ylabel="Hit accuracy",
  897. paired=False, title=period, ylim=[0, 1], colors=[sty.color_dict[gp1][0], sty.color_dict[gp2][0]], det_marker=True, force_markers_identity=True)
  898. ppt.boxplot(ax[1, col], data[data["Genotype"] == gp1]["FPR"], data[data["Genotype"] == gp2]["FPR"], ylabel="Miss error",
  899. paired=False, title=period, ylim=[0, 1], colors=[sty.color_dict[gp1][1], sty.color_dict[gp2][1]], det_marker=False, force_markers_identity=False)
  900. # Plotting the comparisons with shuffled data
  901. ppt.boxplot(ax_shuf[0, col], data[data["Genotype"] == gp1]["TPR"].values, data[data["Genotype"] == gp1]["TPR_shuffle"].values, ylabel="Hit accuracy",
  902. paired=True, title=f"{period}\n{gp1}: Real vs. Shuffled", ylim=[0, 1], colors=[sty.color_dict[gp1][0], "darkgray"], det_marker=True, force_markers_identity=True)
  903. ppt.boxplot(ax_shuf[1, col], data[data["Genotype"] == gp1]["FPR"].values, data[data["Genotype"] == gp1]["FPR_shuffle"].values, ylabel="Miss error",
  904. paired=True, title=f"{period}\n{gp1}: Real vs. Shuffled", ylim=[0, 1], colors=[sty.color_dict[gp1][1], "gray"], det_marker=False, force_markers_identity=True)
  905. ppt.boxplot(ax_shuf[2, col], data[data["Genotype"] == gp2]["TPR"].values, data[data["Genotype"] == gp2]["TPR_shuffle"].values, ylabel="Hit accuracy",
  906. paired=True, title=f"{period}\n{gp2}: Real vs. Shuffled", ylim=[0, 1], colors=[sty.color_dict[gp2][0], "darkgray"], det_marker=True, force_markers_identity=True)
  907. ppt.boxplot(ax_shuf[3, col], data[data["Genotype"] == gp2]["FPR"].values, data[data["Genotype"] == gp2]["FPR_shuffle"].values, ylabel="Miss error",
  908. paired=True, title=f"{period}\n{gp2}: Real vs. Shuffled", ylim=[0, 1], colors=[sty.color_dict[gp2][1], "gray"], det_marker=False, force_markers_identity=True)
  909. data_dict[period] = data
  910. fig.suptitle(f"Hit accuracy miss error comparison ({gp1}/{gp2})\n{title_precision}", fontsize=20)
  911. fig.canvas.manager.set_window_title(f"Hit_Miss_classif_comp_{gp1}_{gp2}_{title_precision}")
  912. fig_shuf.suptitle(f"Shuffle comparison\n{title_precision}", fontsize=20)
  913. fig_shuf.canvas.manager.set_window_title(f"Hit_Miss_classif_comp_shuffle{title_precision}")
  914. # plt.savefig(f"Z:/Current_members/Ourania_Semelidou/2p/Figures_paper & submissions/202507/14/shuffle_{gp1}_{gp2}.pdf", format="pdf")
  915. # plt.savefig(f"Z:/Current_members/Ourania_Semelidou/2p/Figures_paper & submissions/202507/5_Figure2/shuffle_{gp1}_{gp2}.pdf", format="pdf")
  916. plt.show()
  917. return data_dict
  918. def anova_accuracy(frame_model_df):
  919. """
  920. Perform one-way ANOVA on hit accuracy (TPR) and miss error (FPR)
  921. between genotypes during the stimulus window.
  922. The function selects the `"stim"` period (frames 30–45), averages metrics
  923. across frames for each animal, and tests for genotype effects using ANOVA.
  924. Results are printed to console for both hit accuracy and miss error.
  925. Time windows available in the dataset (but only `"stim"` is used here):
  926. - `"stim"`: full stimulus window (frames 30–45).
  927. - `"start_stim (250ms)"`: early stimulus subwindow (frames 30–37).
  928. - `"end_stim (250ms)"`: late stimulus subwindow (frames 37–45).
  929. - `"pre_stim (200ms)"`: baseline window before stimulus onset (frames 24–30).
  930. Parameters
  931. ----------
  932. frame_model_df : pd.DataFrame
  933. Input DataFrame containing columns:
  934. - "Genotype": categorical labels for each animal (e.g., "WT", "KO-Hypo").
  935. - "ID": animal identifier.
  936. - "Frame": time frame index.
  937. - "TPR", "FPR": classification metrics per frame.
  938. Returns
  939. -------
  940. pd.DataFrame
  941. Aggregated DataFrame with mean values per animal and genotype across the `"stim"` window.
  942. """
  943. period_dict = {"stim": [30, 45], "start_stim (250ms)": [30, 37], "end_stim (250ms)": [37, 45],
  944. "pre_stim (200ms)": [24, 30]}
  945. start, end = period_dict["stim"]
  946. data = frame_model_df[frame_model_df["Frame"].isin(range(start, end))].groupby(["Genotype", "ID"], as_index=False).mean().drop(columns="Frame")
  947. aov_hit = pg.anova(data=data, dv="TPR", between="Genotype")
  948. aov_miss = pg.anova(data=data, dv="FPR", between="Genotype")
  949. print("=== Hit accuracy ===")
  950. print(aov_hit)
  951. print("=== Miss error ===")
  952. print(aov_miss)
  953. return data
  954. def compare_accuracy(recs, random=42):
  955. """
  956. Train logistic regression classifiers on excitatory (EXC), inhibitory (INH),
  957. and combined (EXC+INH) neuronal populations to compare decoding accuracy of stimulus detection.
  958. For each recording, the function extracts mean z-scored activity of EXC and INH neurons
  959. during the stimulus period. Logistic regression is trained on balanced subsets of the
  960. training data using random undersampling to avoid class imbalance bias.
  961. Models are evaluated using 4-fold stratified cross-validation, with accuracies computed for:
  962. - All neurons (EXC + INH).
  963. - EXC neurons only.
  964. - INH neurons only.
  965. Accuracies are aggregated across folds, compared within and across genotypes,
  966. and visualized as boxplots:
  967. - Within-genotype comparisons (all vs. EXC, all vs. INH, EXC vs. INH).
  968. - Between-genotype comparisons (WT vs. KO-Hypo for each neuron subset).
  969. Parameters
  970. ----------
  971. recs : list
  972. List of recording objects, each providing:
  973. - `.get_estimated_activity(zscore=True, n_type="EXC"/"INH", estimator="mean", real_duration=True)`
  974. → matrix of neuronal activity (neurons × time).
  975. - `.detected_stim` → binary stimulus detection labels per trial.
  976. - `.filename` → identifier for the recording.
  977. - `.genotype` → genotype label (e.g., "WT", "KO-Hypo").
  978. random : int, default=42
  979. Random seed for reproducibility in cross-validation splits
  980. and random undersampling.
  981. Returns
  982. -------
  983. pd.DataFrame
  984. Aggregated DataFrame containing mean accuracy values per recording and genotype with columns:
  985. - "Genotype": genotype label.
  986. - "ID": recording identifier.
  987. - "acc_all": accuracy with EXC+INH neurons.
  988. - "acc_exc": accuracy with EXC neurons only.
  989. - "acc_inh": accuracy with INH neurons only.
  990. """
  991. rows = []
  992. skf = StratifiedKFold(n_splits=4, shuffle=True, random_state=random)
  993. for rec in recs:
  994. # Retrieving the neuronal activity
  995. exc_mat = rec.get_estimated_activity(zscore=True, n_type="EXC", estimator="mean", real_duration=True)
  996. inh_mat = rec.get_estimated_activity(zscore=True, n_type="INH", estimator="mean", real_duration=True)
  997. full_mat = np.concatenate([exc_mat, inh_mat], axis=0).T
  998. nb_exc = exc_mat.shape[0]
  999. y = rec.detected_stim
  1000. for fold, (train_idx, test_idx) in enumerate(skf.split(full_mat, y), start=1):
  1001. # X_train, X_test, y_train, y_test = train_test_split(full_mat, y, test_size=0.2, stratify=y, random_state=random)
  1002. X_train, X_test = full_mat[train_idx], full_mat[test_idx]
  1003. y_train, y_test = y[train_idx], y[test_idx]
  1004. X_res, y_res = RandomUnderSampler(random_state=random).fit_resample(X_train, y_train)
  1005. model = LogisticRegression(max_iter=5000, random_state=random)
  1006. # Fitting the model with all neurons
  1007. model.fit(X_res, y_res)
  1008. y_pred_all = model.predict(X_test)
  1009. acc_all = get_metrics(y_test, y_pred_all)["Accuracy"]
  1010. # Fitting the model with EXC neurons
  1011. model.fit(X_res[:, :nb_exc], y_res)
  1012. y_pred_exc = model.predict(X_test[:, :nb_exc])
  1013. acc_exc = get_metrics(y_test, y_pred_exc)["Accuracy"]
  1014. # Fitting the model with INH neurons
  1015. model.fit(X_res[:, nb_exc:], y_res)
  1016. y_pred_inh = model.predict(X_test[:, nb_exc:])
  1017. acc_inh = get_metrics(y_test, y_pred_inh)["Accuracy"]
  1018. rows.append({"ID": rec.filename, "Genotype": rec.genotype, "Fold": fold, "acc_all": acc_all, "acc_exc": acc_exc, "acc_inh": acc_inh})
  1019. full_data = pd.DataFrame(rows)
  1020. data = full_data.groupby(["Genotype", "ID"], as_index=False).mean()
  1021. fig, ax = plt.subplots(nrows=3, ncols=4, figsize=(24, 24), constrained_layout=True)
  1022. for row, (group, gp_label) in enumerate(zip([data, data[data["Genotype"] == "WT"], data[data["Genotype"] == "KO-Hypo"]],
  1023. ["all", "WT", "KO-Hypo"])):
  1024. ppt.boxplot(ax[row, 0], group["acc_all"].values, group["acc_exc"].values, ylabel="Accuracy", paired=True, title=f"All neurons/EXC ({gp_label})", ylim=[0, 1],
  1025. colors=[sty.exc_inh_color, sty.exc_color], det_marker=False, force_markers_identity=False)
  1026. ppt.boxplot(ax[row, 1], group["acc_all"].values, group["acc_inh"].values, ylabel="Accuracy", paired=True, title=f"All neurons/INH ({gp_label})", ylim=[0, 1],
  1027. colors=[sty.exc_inh_color, sty.inh_color], det_marker=False, force_markers_identity=False)
  1028. ppt.boxplot(ax[row, 2], group["acc_exc"].values, group["acc_inh"].values, ylabel="Accuracy", paired=True, title=f"EXC/INH ({gp_label})", ylim=[0, 1],
  1029. colors=[sty.exc_color, sty.inh_color], det_marker=False, force_markers_identity=False)
  1030. wt = data[data["Genotype"] == "WT"]
  1031. hypo = data[data["Genotype"] == "KO-Hypo"]
  1032. for row, acc_type in enumerate(["acc_all", "acc_exc", "acc_inh"]):
  1033. ppt.boxplot(ax[row, 3], wt[acc_type].values, hypo[acc_type].values, ylabel="Accuracy", paired=False, title=f"WT/KO-Hypo ({acc_type})", ylim=[0, 1],
  1034. colors=[sty.wt_color, sty.hypo_color], det_marker=False, force_markers_identity=False)
  1035. fig.suptitle("Comparison of decoding accuracy between neuron types")
  1036. fig.canvas.manager.set_window_title(f"Accuracy n_types")
  1037. # plt.show()
  1038. return data
  1039. def correlate_nb_accuracy(recs, accuracy_df, threshold="median"):
  1040. """
  1041. Correlate decoding accuracy with the number of neurons (EXC, INH, and total)
  1042. and compare neuron counts between high- and low-performing animals.
  1043. For each recording, the function retrieves the number of excitatory (EXC) and
  1044. inhibitory (INH) neurons and adds them to the accuracy DataFrame. Linear regression
  1045. analyses are performed between the number of neurons and decoding accuracy
  1046. for EXC, INH, and combined populations. The plots include scatter points grouped
  1047. by genotype, regression lines, and annotations of R² and p-values.
  1048. Additionally, the function splits animals into “low” and “high” performers
  1049. based on an accuracy threshold (median, mean, middle of the range, or a custom float).
  1050. Boxplots compare the number of neurons between low- and high-accuracy groups
  1051. for each metric (acc_exc, acc_inh, acc_all).
  1052. Parameters
  1053. ----------
  1054. recs : dict
  1055. Dictionary mapping recording IDs to recording objects. Each recording object must provide:
  1056. - `.zscore_exc.shape[0]`: number of excitatory neurons.
  1057. - `.zscore_inh.shape[0]`: number of inhibitory neurons.
  1058. accuracy_df : pd.DataFrame
  1059. DataFrame with decoding accuracy results per recording.
  1060. Must include columns:
  1061. - "ID": recording identifier (matching keys in `recs`).
  1062. - "Genotype": genotype label.
  1063. - "acc_exc": decoding accuracy using EXC neurons.
  1064. - "acc_inh": decoding accuracy using INH neurons.
  1065. - "acc_all": decoding accuracy using all neurons.
  1066. threshold : {"median", "mean", "middle"} or float, default="median"
  1067. Method for splitting recordings into low- and high-accuracy groups:
  1068. - "median": median accuracy across animals.
  1069. - "mean": mean accuracy across animals.
  1070. - "middle": midpoint between min and max accuracy.
  1071. - float: user-defined threshold value.
  1072. Returns
  1073. -------
  1074. pd.DataFrame
  1075. Updated accuracy DataFrame including new columns:
  1076. - "n_EXC": number of excitatory neurons per recording.
  1077. - "n_INH": number of inhibitory neurons per recording.
  1078. - "n_all": total number of neurons per recording.
  1079. """
  1080. accuracy_df["n_EXC"] = accuracy_df["ID"].map(lambda id_: recs[id_].zscore_exc.shape[0])
  1081. accuracy_df["n_INH"] = accuracy_df["ID"].map(lambda id_: recs[id_].zscore_inh.shape[0])
  1082. accuracy_df["n_all"] = accuracy_df["n_EXC"] + accuracy_df["n_INH"]
  1083. def plot_lin_reg(ax, data, x_col=None, y_col=None, id_col="ID", group_col=None,
  1084. title=None, xlab=None, ylab=None, colors=None, line_color="red", id_display=True):
  1085. if colors is None:
  1086. colors = {"WT": sty.wt_color, "KO": sty.ko_color, "KO-Hypo": sty.hypo_color}
  1087. xlab = xlab or x_col
  1088. ylab = ylab or y_col
  1089. # Correlation
  1090. results = dict(linregress(data[x_col], data[y_col])._asdict())
  1091. r2 = results["rvalue"] ** 2
  1092. line = results["slope"] * data[x_col] + results["intercept"]
  1093. # Plot the data points and regression line
  1094. ax.plot(data[x_col], line, color=line_color, lw=2)
  1095. if group_col is not None:
  1096. for g in sorted(data[group_col].unique()):
  1097. group = data[data[group_col] == g]
  1098. sc = ax.scatter(group[x_col], group[y_col], color=colors[g], alpha=0.7, label=g, s=10, marker="+")
  1099. if id_display:
  1100. # Save the IDs for this group so that they can be accessed in the callback.
  1101. ids = group[id_col].values
  1102. mplcursors.cursor(sc, hover=True).connect("add", lambda sel, ids=ids: (sel.annotation.set_text(f"ID: {ids[sel.index]}"), sel.annotation.set_fontsize(8)))
  1103. else:
  1104. sc = ax.scatter(data[x_col], data[y_col], color=colors[g], alpha=0.7, label=g, s=10, marker="+")
  1105. if id_display:
  1106. ids = data["ID"].values
  1107. mplcursors.cursor(sc, hover=True).connect("add", lambda sel, ids=ids: (sel.annotation.set_text(f"ID: {ids[sel.index]}"), sel.annotation.set_fontsize(8)))
  1108. # Annotate the plot with R² and p-value
  1109. ax.text(0.05, 0.95, f"$r^2 = {r2:.3f}$\np-value = {results["pvalue"]:.3f}", transform=ax.transAxes, fontsize=8, verticalalignment="top", color="black")
  1110. ax.set_title(title, fontsize=12)
  1111. ax.set_xlabel(xlab, fontsize=10)
  1112. ax.set_ylabel(ylab, fontsize=10)
  1113. for lbl in ax.get_xticklabels() + ax.get_yticklabels():
  1114. lbl.set_fontsize(8)
  1115. return results
  1116. fig, ax = plt.subplots(nrows=4, ncols=3, figsize=(18, 24), constrained_layout=True)
  1117. plot_lin_reg(ax[0 ,0], accuracy_df, x_col="n_EXC", y_col="acc_exc", group_col="Genotype", title="EXC")
  1118. plot_lin_reg(ax[0 ,1], accuracy_df, x_col="n_INH", y_col="acc_inh", group_col="Genotype", title="INH")
  1119. plot_lin_reg(ax[0 ,2], accuracy_df, x_col="n_all", y_col="acc_all", group_col="Genotype", title="All")
  1120. # Comparing the number of neurons between individuals with high and low accuracy
  1121. for col, accuracy_metric in enumerate(["acc_exc", "acc_inh", "acc_all"]):
  1122. if threshold == "median":
  1123. t_val = np.percentile(accuracy_df[accuracy_metric].values, 50)
  1124. elif threshold == "mean":
  1125. t_val = np.mean(accuracy_df[accuracy_metric].values)
  1126. elif threshold == "middle":
  1127. t_val = (max(accuracy_df[accuracy_metric].values) + min(accuracy_df[accuracy_metric].values))/2
  1128. elif isinstance(threshold, float):
  1129. t_val = threshold
  1130. ax[0, col].axhline(y=t_val, linestyle="--", color="gray", lw=0.5)
  1131. low_perf = accuracy_df[accuracy_df[accuracy_metric] < t_val]
  1132. high_perf = accuracy_df[accuracy_df[accuracy_metric] >= t_val]
  1133. ppt.boxplot(ax[1, col], low_perf["n_EXC"].values, high_perf["n_EXC"].values, ylabel="n_EXC", paired=False,
  1134. title=f"{accuracy_metric} Low/High acc", ylim=[],
  1135. colors=[sty.exc_color, sty.exc_color], det_marker=False, force_markers_identity=False)
  1136. ppt.boxplot(ax[2, col], low_perf["n_INH"].values, high_perf["n_INH"].values, ylabel="n_INH", paired=False,
  1137. title=f"{accuracy_metric} Low/High acc", ylim=[],
  1138. colors=[sty.inh_color, sty.inh_color], det_marker=False, force_markers_identity=False)
  1139. ppt.boxplot(ax[3, col], low_perf["n_all"].values, high_perf["n_all"].values, ylabel="n_all", paired=False,
  1140. title=f"{accuracy_metric} Low/High acc", ylim=[],
  1141. colors=[sty.exc_inh_color, sty.exc_inh_color], det_marker=False, force_markers_identity=False)
  1142. fig.suptitle(f"Correlation of the model accuracy with the number of neurons.\nComparison of the number of neurons "
  1143. f"between high and low accuracy animal (threshold={threshold})", fontsize=12)
  1144. fig.canvas.manager.set_window_title("Corr acc nb neurons")
  1145. plt.show()
  1146. return accuracy_df
  1147. def correlate_frame_accuracy_metrics(frame_model_df, features_df, period="stim"):
  1148. """
  1149. Correlates model accuracy during a specific time period with neuronal features to assess
  1150. which features carry the most information about decoding performance.
  1151. The function computes the mean model accuracy (TPR, FPR, and overall Accuracy) over
  1152. a selected period (`stim`, `start`, `end`, or `pre_stim`), then matches these accuracies
  1153. with averaged neuronal features. Only responsive neurons (amplitude != 0) are considered
  1154. when extracting features. Two types of feature DataFrames are created:
  1155. - behavior-specific (features computed separately for hit and miss trials),
  1156. - global (features aggregated across all trials for each animal).
  1157. These features are then correlated with model accuracy metrics, both globally and within
  1158. genotypes (WT and KO-Hypo). Scatter plots with regression lines and R² / p-value annotations
  1159. are generated for each feature-accuracy pair across trial types (All, Hit, Miss).
  1160. Parameters
  1161. ----------
  1162. frame_model_df : pandas.DataFrame
  1163. DataFrame containing frame-wise model results with columns such as
  1164. "Frame", "Genotype", "ID", "TPR", "FPR", "Accuracy".
  1165. features_df : pandas.DataFrame
  1166. DataFrame containing neuronal features with columns like
  1167. "ID", "Genotype", "behavior", "bounded_x0", "amplitude", "threshold", etc.
  1168. Only neurons with nonzero amplitude are considered.
  1169. period : str, optional
  1170. The time window over which model accuracy is averaged. Options are:
  1171. - "stim" (default) : frames 30–45
  1172. - "start" : frames 30–37
  1173. - "end" : frames 37–45
  1174. - "pre_stim" : frames 24–30
  1175. Returns
  1176. -------
  1177. data : pandas.DataFrame
  1178. Merged DataFrame of averaged features and model accuracy for each animal,
  1179. with behavior-specific separation where applicable.
  1180. results : pandas.DataFrame
  1181. Summary of correlation statistics (R² and p-values) for each feature-accuracy pair,
  1182. computed globally, for WT only, and for KO-Hypo only. This can be used to identify
  1183. which neuronal features explain variance in decoding accuracy.
  1184. """
  1185. # Getting a Dataframe of the model mean accuracy during stim, one row per animal
  1186. period_dict = {"stim": [30, 45], "start": [30, 37], "end": [37, 45], "pre_stim": [24, 30]}
  1187. start, end = period_dict[period]
  1188. data_acc = frame_model_df[frame_model_df["Frame"].isin(range(start, end))].groupby(["Genotype", "ID"], as_index=False).mean().drop(columns="Frame")
  1189. # Getting the features DataFrame, defining a single metric per animal, only responsive neurons are considered
  1190. data_features = features_df[features_df.amplitude != 0].groupby(["ID", "Genotype", "behavior"], as_index=False).mean().drop(columns=["bounded_x0", "amplitude", "threshold"])
  1191. data_features_all = features_df[features_df.amplitude != 0].drop(columns="behavior").groupby(["ID", "Genotype"], as_index=False).mean().drop(columns=["bounded_x0", "amplitude", "threshold"])
  1192. data = data_features.merge(data_acc[["ID", "TPR", "FPR", "Accuracy"]], on="ID", how="left")
  1193. data_hit = data[data.behavior == True]
  1194. data_miss = data[data.behavior == False]
  1195. data_all = data_features_all.merge(data_acc[["ID", "TPR", "FPR", "Accuracy"]], on="ID", how="left")
  1196. cols_id = ["ID", "Genotype", "behavior"]
  1197. cols_acc = ["Accuracy", "TPR", "FPR"]
  1198. cols_features = [col for col in data.columns if ((col not in cols_id) and (col not in cols_acc))]
  1199. # Plotting the correlation of the different features with the model accuracy for all genotypes
  1200. fig, ax = plt.subplots(nrows=9, ncols=20, figsize=(100, 45), constrained_layout=True)
  1201. rows = []
  1202. for row, (data, trial_type) in enumerate(zip([data_all, data_hit, data_miss], ["All", "Hit", "Miss"])):
  1203. for sub_row, acc in enumerate(cols_acc):
  1204. for col, feature in enumerate(cols_features):
  1205. # Correlation global
  1206. y_col = data[acc]
  1207. x_col = data[feature]
  1208. results = dict(linregress(x_col, y_col)._asdict())
  1209. r2 = results["rvalue"] ** 2
  1210. line = results["slope"] * x_col + results["intercept"]
  1211. # Correlation WT
  1212. y_col_wt = data[data.Genotype == "WT"][acc]
  1213. x_col_wt = data[data.Genotype == "WT"][feature]
  1214. results_wt = dict(linregress(x_col_wt, y_col_wt)._asdict())
  1215. r2_wt = results_wt["rvalue"] ** 2
  1216. line_wt = results_wt["slope"] * x_col_wt + results_wt["intercept"]
  1217. # Correlation KO-Hypo
  1218. y_col_hypo = data[data.Genotype == "KO-Hypo"][acc]
  1219. x_col_hypo = data[data.Genotype == "KO-Hypo"][feature]
  1220. results_hypo = dict(linregress(x_col_hypo, y_col_hypo)._asdict())
  1221. r2_hypo = results_hypo["rvalue"] ** 2
  1222. line_hypo = results_hypo["slope"] * x_col_hypo + results_hypo["intercept"]
  1223. # Plot the data points and regression lines
  1224. ax[3 * row + sub_row, col].plot(x_col, line, color="black", lw=2)
  1225. ax[3 * row + sub_row, col].plot(x_col_wt, line_wt, color=sty.wt_color, lw=2)
  1226. ax[3 * row + sub_row, col].plot(x_col_hypo, line_hypo, color=sty.hypo_color, lw=2)
  1227. for g in sorted(data["Genotype"].unique()):
  1228. group = data[data["Genotype"] == g]
  1229. sc = ax[3 * row + sub_row, col].scatter(group[feature], group[acc], color=sty.color_dict[g][0], alpha=0.7, s=10, marker="+")
  1230. # Annotate the plot with R² and p-value
  1231. ax[3 * row + sub_row, col].text(0.05, 0.95, f"$r^2={r2:.3f}$ p-val={results["pvalue"]:.3f}", transform=ax[3 * row + sub_row, col].transAxes, fontsize=8, verticalalignment="top", color="black")
  1232. ax[3 * row + sub_row, col].text(0.05, 0.90, f"$r^2={r2_wt:.3f}$ p-val={results_wt["pvalue"]:.3f}", transform=ax[3 * row + sub_row, col].transAxes, fontsize=8, verticalalignment="top", color=sty.wt_color)
  1233. ax[3 * row + sub_row, col].text(0.05, 0.85, f"$r^2={r2_hypo:.3f}$ p-val={results_hypo["pvalue"]:.3f}", transform=ax[3 * row + sub_row, col].transAxes, fontsize=8, verticalalignment="top", color=sty.hypo_color)
  1234. ax[3 * row + sub_row, col].set_title(f"{trial_type} trials", fontsize=10)
  1235. ax[3 * row + sub_row, col].set_xlabel(feature, fontsize=10)
  1236. ax[3 * row + sub_row, col].set_ylabel(acc, fontsize=10)
  1237. ax[3 * row + sub_row, col].set_ylim(ymin=0, ymax=1)
  1238. ax[3 * row + sub_row, col].tick_params(axis='both', which='major', labelsize=5)
  1239. rows.append({"Trials": trial_type, "Acc_metric": acc, "Feature": feature,
  1240. "r2": r2, "pval": results["pvalue"],
  1241. "r2_wt": r2_wt, "pval_wt": results_wt["pvalue"],
  1242. "r2_hypo": r2_hypo, "pval_hypo": results_hypo["pvalue"]})
  1243. results = pd.DataFrame(rows)
  1244. fig.suptitle(f"Correlation of the frame model accuracy during the whole {period} period with the different metrics computed on responsive neurons")
  1245. # plt.savefig(f"{server_address}/response_decoding/corr_acc_model_features_{period}.pdf", format="pdf")
  1246. return data, results
  1247. def correlate_frame_accuracy_ntn_df(frame_model_df, ntn_df, period="stim"):
  1248. """
  1249. Correlates frame-wise model accuracy with neuron-to-neuron cosine similarity (ntn_cosim) metrics.
  1250. For each animal, the mean model accuracy (TPR, FPR, and overall Accuracy) is computed over a
  1251. specified period (`stim`, `start`, `end`, or `pre_stim`). These accuracies are then merged
  1252. with the ntn_cosim metrics from `ntn_df`. Data is separated into trial types: "All", "Hit",
  1253. and "Miss".
  1254. Correlation analyses are performed for each feature in `ntn_df`:
  1255. - Globally across all animals.
  1256. - Within WT genotype only.
  1257. - Within KO-Hypo genotype only.
  1258. Linear regression lines are plotted, and R² and p-values are annotated for each feature-accuracy pair.
  1259. Scatter points are colored by genotype.
  1260. Parameters
  1261. ----------
  1262. frame_model_df : pandas.DataFrame
  1263. DataFrame containing frame-wise model results with columns such as
  1264. "Frame", "Genotype", "ID", "TPR", "FPR", "Accuracy".
  1265. ntn_df : pandas.DataFrame
  1266. DataFrame containing neuron-to-neuron cosine similarity metrics, with columns
  1267. including "ID", "Genotype", "Behavior", and one or more feature columns.
  1268. period : str, optional
  1269. Time window over which model accuracy is averaged. Options:
  1270. - "stim" (default) : frames 30–45
  1271. - "start" : frames 30–37
  1272. - "end" : frames 37–45
  1273. - "pre_stim" : frames 24–30
  1274. Returns
  1275. -------
  1276. data : pandas.DataFrame
  1277. Merged DataFrame of ntn_cosim features and model accuracy for each animal, separated by
  1278. trial type.
  1279. results : pandas.DataFrame
  1280. Summary of correlation statistics (R² and p-values) for each feature-accuracy pair,
  1281. computed globally, for WT only, and for KO-Hypo only.
  1282. """
  1283. # Getting a Dataframe of the model mean accuracy during stim, one row per animal
  1284. period_dict = {"stim": [30, 45], "start": [30, 37], "end": [37, 45], "pre_stim": [24, 30]}
  1285. start, end = period_dict[period]
  1286. data_acc = frame_model_df[frame_model_df["Frame"].isin(range(start, end))].groupby(["Genotype", "ID"], as_index=False).mean().drop(columns="Frame")
  1287. # Getting the features DataFrame, defining a single metric per animal, only responsive neurons are considered
  1288. data = ntn_df.merge(data_acc[["ID", "TPR", "FPR", "Accuracy"]], on="ID", how="left").drop(columns=["Threshold"])
  1289. data_hit = data[data.Behavior == "Hit"]
  1290. data_miss = data[data.Behavior == "Miss"]
  1291. data_all = data[data.Behavior == "All"]
  1292. cols_id = ["ID", "Genotype", "Behavior"]
  1293. cols_acc = ["Accuracy", "TPR", "FPR"]
  1294. cols_features = [col for col in data.columns if ((col not in cols_id) and (col not in cols_acc))]
  1295. n_features = len(cols_features)
  1296. # Plotting the correlation of the different features with the model accuracy for all genotypes
  1297. fig, ax = plt.subplots(nrows=9, ncols=n_features, figsize=(n_features * 5, 45), constrained_layout=True)
  1298. rows = []
  1299. for row, (data, trial_type) in enumerate(zip([data_all, data_hit, data_miss], ["All", "Hit", "Miss"])):
  1300. for sub_row, acc in enumerate(cols_acc):
  1301. for col, feature in enumerate(cols_features):
  1302. # Correlation global
  1303. y_col = data[acc]
  1304. x_col = data[feature]
  1305. results = dict(linregress(x_col, y_col)._asdict())
  1306. r2 = results["rvalue"] ** 2
  1307. line = results["slope"] * x_col + results["intercept"]
  1308. # Correlation WT
  1309. y_col_wt = data[data.Genotype == "WT"][acc]
  1310. x_col_wt = data[data.Genotype == "WT"][feature]
  1311. results_wt = dict(linregress(x_col_wt, y_col_wt)._asdict())
  1312. r2_wt = results_wt["rvalue"] ** 2
  1313. line_wt = results_wt["slope"] * x_col_wt + results_wt["intercept"]
  1314. # Correlation KO-Hypo
  1315. y_col_hypo = data[data.Genotype == "KO-Hypo"][acc]
  1316. x_col_hypo = data[data.Genotype == "KO-Hypo"][feature]
  1317. results_hypo = dict(linregress(x_col_hypo, y_col_hypo)._asdict())
  1318. r2_hypo = results_hypo["rvalue"] ** 2
  1319. line_hypo = results_hypo["slope"] * x_col_hypo + results_hypo["intercept"]
  1320. # Defining the axis
  1321. if n_features == 1:
  1322. axis = ax[3 * row + sub_row]
  1323. else:
  1324. axis = ax[3 * row + sub_row, col]
  1325. # Plot the data points and regression lines
  1326. axis.plot(x_col, line, color="black", lw=2)
  1327. axis.plot(x_col_wt, line_wt, color=sty.wt_color, lw=2)
  1328. axis.plot(x_col_hypo, line_hypo, color=sty.hypo_color, lw=2)
  1329. for g in sorted(data["Genotype"].unique()):
  1330. group = data[data["Genotype"] == g]
  1331. sc = axis.scatter(group[feature], group[acc], color=sty.color_dict[g][0], alpha=0.7, s=10, marker="+")
  1332. # Annotate the plot with R² and p-value
  1333. axis.text(0.05, 0.95, f"$r^2={r2:.3f}$ p-val={results["pvalue"]:.3f}", transform=axis.transAxes, fontsize=8, verticalalignment="top", color="black")
  1334. axis.text(0.05, 0.90, f"$r^2={r2_wt:.3f}$ p-val={results_wt["pvalue"]:.3f}", transform=axis.transAxes, fontsize=8, verticalalignment="top", color=sty.wt_color)
  1335. axis.text(0.05, 0.85, f"$r^2={r2_hypo:.3f}$ p-val={results_hypo["pvalue"]:.3f}", transform=axis.transAxes, fontsize=8, verticalalignment="top", color=sty.hypo_color)
  1336. axis.set_title(f"{trial_type} trials", fontsize=10)
  1337. axis.set_xlabel(feature, fontsize=10)
  1338. axis.set_ylabel(acc, fontsize=10)
  1339. axis.set_ylim(ymin=0, ymax=1)
  1340. axis.tick_params(axis='both', which='major', labelsize=5)
  1341. rows.append({"Trials": trial_type, "Acc_metric": acc, "Feature": feature,
  1342. "r2": r2, "pval": results["pvalue"],
  1343. "r2_wt": r2_wt, "pval_wt": results_wt["pvalue"],
  1344. "r2_hypo": r2_hypo, "pval_hypo": results_hypo["pvalue"]})
  1345. results = pd.DataFrame(rows)
  1346. fig.suptitle(f"Correlation of the frame model accuracy during the whole {period} period with neuron to neuron global cosine similarity")
  1347. # plt.savefig(f"{server_address}response_decoding/corr_acc_model_ntn_cosim_{period}.pdf", format="pdf")
  1348. return data, results
  1349. # endregion ============================================================================================================
  1350. if __name__ == '__main__':
  1351. BMS_analysis = True
  1352. ### Initialisation of recs instances ###
  1353. if BMS_analysis:
  1354. directory = "C:/Users/cvandromme/Desktop/Tactile_detection/Data_DMSO_BMS/"
  1355. roi_path = "C:/Users/cvandromme/Desktop/Tactile_detection/Fmko_bms&dmso_info.xlsx"
  1356. else:
  1357. directory = "C:/Users/cvandromme/Desktop/Tactile_detection/Data/"
  1358. roi_path = "C:/Users/cvandromme/Desktop/Tactile_detection/FmKO_ROIs&inhibitory.xlsx"
  1359. server_address = "Z:/Current_members/Ourania_Semelidou/2p/Figures_paper & submissions/Figures_april_2025/"
  1360. roi_info = pd.read_excel(roi_path)
  1361. files = os.listdir(directory)
  1362. files_ = [file for file in files if file.endswith("synchro")]
  1363. def opening_rec(fil, i):
  1364. rec = pc.RecordingAmplDet(directory + fil + "/", 0, roi_path)
  1365. return rec
  1366. workers = cpu_count()
  1367. pool = pool.ThreadPool(processes=workers)
  1368. async_results = [pool.apply_async(opening_rec, args=(file, i)) for i, file in enumerate(files_)]
  1369. if BMS_analysis:
  1370. recs = {f"{ar.get().filename}-{ar.get().genotype.split("-")[1]}": ar.get() for ar in async_results}
  1371. else:
  1372. recs = {ar.get().filename: ar.get() for ar in async_results}
  1373. for rec in recs.values():
  1374. # rec.auc_neg()
  1375. rec.peak_delay_amp()
  1376. # endregion ============
  1377. # test = perceptual_magnitude_behavior_corr(recs.values())
  1378. # mean_corr = test.groupby("Genotype").mean()
  1379. # glmm_behavior(data)
  1380. # frame_data = get_activity_by_frame_df(recs.values(), zscore=True)
  1381. # corr_data = correlate_mean_zscore_behavior_frame(frame_data[frame_data["Amplitude"] == 12])
  1382. # plot_frame_correlation(corr_data)
  1383. frame_dff = get_activity_by_frame_df(recs.values(), zscore=False, BMS=BMS_analysis)
  1384. # features_df = get_features(recs.values(), amp_delay=True, auc=True)
  1385. # activity_long_df = get_mean_trial_activity_df(recs.values(), zscore=True)
  1386. # ntn_df = ntn_cosine_similarity(activity_long_df, amplitude="all", nogo=False, metric="Resp", filter_out_nr=False)
  1387. # frame_dff_3 = aggregate_every_3cols(frame_dff, ["Genotype", "ID", "Threshold", "Trial", "Amplitude", "Duration", "Behavior", "n_type", "resp", "n_ID"], window_size=3)
  1388. frame_model_df_res = pd.read_csv("Z:/Current_members/Ourania_Semelidou/2p/Figures_paper & submissions/202507/5_Figure2/frame_model_df_res.csv")
  1389. # frame_model_df_avg = pd.read_csv("C:/Users/cvandromme/Desktop/Tactile_detection/Analysis_data/frame_model_df_avg3.csv")
  1390. # frame_model_df_bms, frame_mean_sem_bms = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling")
  1391. # frame_model_df_res, frame_mean_sem_res = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling")
  1392. # frame_model_df_avg, frame_mean_sem_avg = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling", sliding_window=3, window_sum=False)
  1393. # frame_model_df_3, frame_mean_sem_3 = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling", window=3)
  1394. # frame_model_df_5, frame_mean_sem_5 = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling", window=5)
  1395. # frame_model_df_2, frame_mean_sem_2 = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling", window=2)
  1396. # frame_model_df_4, frame_mean_sem_4 = frame_model(frame_dff, neuron_type=["EXC", "INH"], resp_type=[0, 1, -1], db_cv=True, balancing_method="resampling")
  1397. plot_df = plot_hit_miss_classif(frame_model_df_res, title_precision="All neurons, all patterns, doubleCV", shuffle=True, timescale_division_factor=1)
  1398. data_comp = plot_hit_miss_classif_comp(frame_model_df_res, gp1="WT", gp2="KO", title_precision="All neurons, all patterns, doubleCV")
  1399. # plot_df = plot_hit_miss_classif(frame_model_df_4, title_precision="All neurons, all patterns, doubleCV, non-sliding avg4", shuffle=True, timescale_division_factor=4)
  1400. plot_hit_miss_classif_comp(frame_model_df_bms, gp1="WT-DMSO", gp2="WT-BMS", title_precision="All neurons, all patterns, doubleCV")
  1401. plot_hit_miss_classif_comp(frame_model_df_bms, gp1="KO-DMSO", gp2="KO-BMS", title_precision="All neurons, all patterns, doubleCV")
  1402. data_comp = plot_hit_miss_classif_comp(frame_model_df_bms, gp1="WT-DMSO", gp2="KO-DMSO", title_precision="All neurons, all patterns, doubleCV")
  1403. plot_hit_miss_classif_comp(frame_model_df_bms, gp1="WT-BMS", gp2="KO-BMS", title_precision="All neurons, all patterns, doubleCV")
  1404. # corr_acc_features_df, corr_results_df = correlate_frame_accuracy_metrics(frame_model_df_res, features_df, period="stim")
  1405. # corr_acc_ntn_df, corr_ntn_results_df = correlate_frame_accuracy_ntn_df(frame_model_df_res, ntn_df, period="stim")
  1406. # saved_framed_model_df = pd.read_csv("C:/Users/cvandromme/Desktop/frame_model_df.csv")
  1407. # frame_model_df_amp_gp = frame_model_df.groupby(["Genotype", "ID", "Threshold", "Amplitude"], as_index=False).mean().drop(columns=["Trial", "Duration", "Behavior"])
  1408. # plot_hit_miss_classif(saved_framed_model_df)
  1409. # data = plot_hit_miss_classif_comp(saved_framed_model_df, gp1="WT", gp2="KO", title_precision="['EXC',_'INH'][0,_1,_-1]_-_db_cv=True")
  1410. # recs_wt = {k: v for k, v in recs.items() if v.genotype == "WT"}
  1411. # recs_hypo = {k: v for k, v in recs.items() if v.genotype == "KO-Hypo"}
  1412. accuracy_comp_df = compare_accuracy(recs.values(), random=42)
  1413. # acc_nb_df = correlate_nb_accuracy(recs, accuracy_comp_df[accuracy_comp_df["Genotype"] == "KO"], threshold="median")

response_decoding.py at commit 7db6cf1, under GPL-2.0 · at the source

Overview

Authors: Ourania Semelidou1, Théo Gauvrit1, Célien Vandromme1, Alexandre Cornier1, Anna Saint‐Jean1, Yves Le Feuvre1, Melanie Ginger1, Andreas Frick1
  1. University of Bordeaux, INSERM, Neurocentre Magendie, Bordeaux, France
Institutions: Université de Bordeaux (France); Inserm (France); Neurocentre Magendie (France)
Journal: Advanced science (Weinheim, Baden-Wurttemberg, Germany), volume 13, issue 28, article e19479
Dates: received 2 October 2025; accepted 1 January 2026; published online 15 April 2026; in print May 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1002/advs.202519479 · PMID 41983266 · PMCID PMC13185836 · OpenAlex W7154458527
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: optical imaging (calcium, voltage, 2-photon) (modality), human (organism), mouse (organism), other condition (population), autism (population), cognitive (subfield)
Methods: Statistics, Machine learning, Evoked potentials, fMRI & imaging, Single-unit activity, calcium imaging, Smoothing, state filtering, decompositions
Keywords: calcium imaging, hyposensitivity, mouse model of autism, neural mechanisms, perceptual decision‐making, signal‐to‐noise ratio, tactile perception
MeSH: Autistic Disorder*, Fragile X Messenger Ribonucleoprotein 1*, Somatosensory Cortex*, Touch*, Touch Perception*, Animals, Disease Models, Animal, Humans, Male, Mice, Neurons, Signal-To-Noise Ratio (* major topic)
Topic: Autism Spectrum Disorder Research (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: Marcel Dassault-Fondation FondaMental Award 2019; Brain and Behavior Research Foundation Young Investigator (32694); Foundation for Medical Research (FRM) postdoc fellowship (SPF202005011992); ANR-MultiSens; Simons Foundation Autism Research Initiative; NeuroCentre Magendie (INSERM U1215); Foundation for Brain Research (FRC); ANR-EMT-Sens; Agence Nationale de la Recherche (ANR-EMT-Sens, ANR‐EMT‐Sens)
Citations: not cited yet (Europe PMC); 104 references in the paper

Abstract

Touch is essential for interacting with the world, and atypical tactile experience is a core feature of autism that profoundly affects daily life. However, we do not know the neural mechanisms of low‐level tactile perception and their alterations in autism. Using a translational forepaw‐based perceptual task, we recapitulate the multifaceted tactile features of autistic individuals in the Fmr1 −/y mouse model of autism, showing reduced detection of low‐level vibrotactile stimuli, interindividual variability, and unreliable responses. We reveal that impaired detection decoding in Fmr1 −/y‐hyposensitive mice stems from diminished single‐neuron signal‐to‐noise ratio within layers 2/3 of the primary somatosensory cortex that contributes to weak population encoding of the tactile stimulus and its detection. This manifests as reduced stimulus‐dependent neural recruitment, impaired response precision, and disrupted ensemble dynamics. Decreasing neuronal excitability strengthens sensory encoding and restores tactile perception. This work provides a translational framework for probing neuronal‐perceptual changes in neurodevelopmental conditions, reveals inter‐individual variability in preclinical models, and uncovers the neural basis of tactile hyposensitivity in autism.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 14 matches between paragraphs and lines of code.

FrickLab/Imaging_2PCalciumAnalysis

License: GPL-2.0
State: the link answers, verified on 29 September 2026
Evidence: files inventoried
Commit: 7db6cf1c509f5f157105c9d610148413b88d5eb7, 14 October 2025
Languages: Python (44), Jupyter (11)
Size: 69 files, 55 scripts
Software Heritage: not archived
Found in: “Code Availability”
Holds: README, license file, environment (requirements.txt), documentation, 11 notebooks
Not found: CITATION.cff, tests, continuous integration
Tools: Matplotlib (43 files), NumPy (43 files), pandas (36 files), SciPy (25 files), scikit-learn (19 files), statsmodels (9 files), Pingouin (6 files), imbalanced-learn (5 files), seaborn (2 files), h5py (1 file)
Availability: 1 check, the latest on 29 September 2026: the link answers
  • 29 September 2026: the link answers
57 files

Code Availability

Custom‐made Python Codes Used in this Study can be found on the following GitHub Repository: https://Github.Com/FrickLab/Imaging_2PCalciumAnalysis

Reproduced under the paper's license (CC BY), from the paper cited above.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 55 scripts, each with its path and the digest of its content;
  • 14 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data Availability Statement

The data that support the findings of this study are openly available in figshare at https://doi.org/10.6084/m9.figshare.29900456, reference number 29900456.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 29 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 8 authors, 7 keywords, 12 MeSH terms, 9 funders, 101 references.

Cite

This paper

Semelidou, O., Gauvrit, T., Vandromme, C., Cornier, A., Saint‐Jean, A., Feuvre, Y. L., Ginger, M., & Frick, A. (2026). Diminished Signal-to-Noise Ratio Disrupts Somatosensory Population Encoding and Drives Tactile Hyposensitivity in the Fmr1&lt;sup&gt;-/y&lt;/sup&gt; Autism Model. Advanced science (Weinheim, Baden-Wurttemberg, Germany), 13(28), e19479. https://doi.org/10.1002/advs.202519479

BibTeX

@article{semelidou2026diminished,
author = {Semelidou, Ourania and Gauvrit, Théo and Vandromme, Célien and Cornier, Alexandre and Saint‐Jean, Anna and Feuvre, Yves Le and Ginger, Melanie and Frick, Andreas},
title = {{Diminished Signal-to-Noise Ratio Disrupts Somatosensory Population Encoding and Drives Tactile Hyposensitivity in the Fmr1\&lt;sup\&gt;-/y\&lt;/sup\&gt; Autism Model}},
journal = {Advanced science (Weinheim, Baden-Wurttemberg, Germany)},
year = {2026},
month = apr,
volume = {13},
number = {28},
pages = {e19479},
publisher = {Wiley},
issn = {2198-3844},
doi = {10.1002/advs.202519479},
url = {https://doi.org/10.1002/advs.202519479},
pmid = {41983266},
pmcid = {PMC13185836}
}

RIS

TY - JOUR
AU - Semelidou, Ourania
AU - Gauvrit, Théo
AU - Vandromme, Célien
AU - Cornier, Alexandre
AU - Saint‐Jean, Anna
AU - Feuvre, Yves Le
AU - Ginger, Melanie
AU - Frick, Andreas
TI - Diminished Signal-to-Noise Ratio Disrupts Somatosensory Population Encoding and Drives Tactile Hyposensitivity in the Fmr1&lt;sup&gt;-/y&lt;/sup&gt; Autism Model
T2 - Advanced science (Weinheim, Baden-Wurttemberg, Germany)
J2 - Adv Sci (Weinh)
PY - 2026
DA - 2026/04/15
VL - 13
IS - 28
SP - e19479
SN - 2198-3844
PB - Wiley
DO - 10.1002/advs.202519479
UR - https://doi.org/10.1002/advs.202519479
LA - en
ER -

CSL-JSON

{
"id": "10.1002/advs.202519479",
"type": "article-journal",
"title": "Diminished Signal-to-Noise Ratio Disrupts Somatosensory Population Encoding and Drives Tactile Hyposensitivity in the Fmr1&lt;sup&gt;-/y&lt;/sup&gt; Autism Model",
"container-title": "Advanced science (Weinheim, Baden-Wurttemberg, Germany)",
"author": [
{
"family": "Semelidou",
"given": "Ourania"
},
{
"family": "Gauvrit",
"given": "Théo"
},
{
"family": "Vandromme",
"given": "Célien"
},
{
"family": "Cornier",
"given": "Alexandre"
},
{
"family": "Saint‐Jean",
"given": "Anna"
},
{
"family": "Feuvre",
"given": "Yves Le"
},
{
"family": "Ginger",
"given": "Melanie"
},
{
"family": "Frick",
"given": "Andreas"
}
],
"container-title-short": "Adv Sci (Weinh)",
"volume": "13",
"issue": "28",
"page": "e19479",
"DOI": "10.1002/advs.202519479",
"PMID": "41983266",
"PMCID": "PMC13185836",
"ISSN": "2198-3844",
"publisher": "Wiley",
"URL": "https://doi.org/10.1002/advs.202519479",
"language": "en",
"issued": {
"date-parts": [
[
2026,
4,
15
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1016/j.isci.2026.116825 [code]
Social hierarchy shapes behavioral and transcriptional responses to chronic stress and ketamine in male mice.
Journal: iScience
In common: imbalanced-learn, Pingouin, h5py, 7 other tools, mouse
[2] doi:10.1016/j.celrep.2026.117590 [code]
Impaired behavioral inhibition in Fmr1 KO mice is linked to disrupted visual cortex theta oscillations.
Journal: Cell reports
In common: Pingouin, statsmodels, seaborn, 5 other tools, autism, other condition, mouse, 1 reference
[3] doi:10.7554/elife.109717 [code]
Retrosplenial cortex enables context-dependent goal-directed sensorimotor transformation.
Journal: eLife
In common: h5py, statsmodels, seaborn, 5 other tools, mouse, 2 references
[4] doi:10.64898/2026.07.14.26358047 [code]
From Genes to Neurochemistry: Excitation and Inhibition Mechanisms of Sensory Differences in Autism
Journal: medRxiv (preprint)
In common: autism, 6 references
[5] doi:10.1016/j.nicl.2026.104012 [code]
Structural-functional multilayer brain network properties and outcome of combined repetitive transcranial magnetic stimulation and psychotherapy for obsessive-compulsive disorder.
Journal: NeuroImage. Clinical
In common: imbalanced-learn, Pingouin, statsmodels, 6 other tools, other condition
[6] doi: [code]
Real-time closed-loop feedback system for mouse mesoscale cortical signal and movement control
Journal: eLife
In common: Pingouin, h5py, statsmodels, 6 other tools, optical imaging (calcium, voltage, 2-photon), mouse
[7] doi:10.1038/s41398-026-04081-8 [code]
Functional system-specific brain aging across the Alzheimer's disease continuum.
Journal: Translational psychiatry
In common: imbalanced-learn, Pingouin, statsmodels, 6 other tools
[8] doi:10.1038/s41598-026-64405-y [code]
Evaluating clinical and neuroimaging predictors for cognitive-behavioral therapy outcome in obsessive-compulsive disorder.
Journal: Scientific reports
In common: imbalanced-learn, h5py, statsmodels, 6 other tools, other condition
[9] doi:10.1038/s41467-026-76939-w [code]
HIPPIE: a generative model for electrophysiological analysis across species, technologies, and modalities.
Journal: Nature communications
In common: imbalanced-learn, h5py, statsmodels, 6 other tools, mouse
[10] doi:10.1038/s41593-026-02362-5 [code]
Replay of procedural memory is independent of the hippocampus.
Journal: Nature neuroscience
In common: Pingouin, h5py, statsmodels, 6 other tools, cognitive, mouse

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.