OSCR

Intrinsic timing, not temporal prediction, underlies ramping dynamics in visual and parietal cortex during passive behavior.

Code ↔ Paper

2 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 2 matches
  1. [1] § MATERIALS AND METHODS › Population trajectory analysis ↔ 2AFC/Modules/reader.py, lines 643–699 · score 0.56 · Savitzky Golay filtering, polynomial, window, neural
  2. [2] § MATERIALS AND METHODS › Population trajectory analysis ↔ 2p_2AFC_reg_version/Modules/reader.py, lines 640–696 · score 0.56 · Savitzky Golay filtering, polynomial, window, neural

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 706 lines · 35 KB · no license · 1 match

  1. # Liberary for reading session data from multiple directories
  2. import os
  3. import numpy as np
  4. import h5py
  5. import shutil
  6. import pandas as pd
  7. import scipy.io as sio
  8. from scipy.signal import savgol_filter
  10. # First fuction to read for reading sessions data from multiple directories
  11. def read_ops(list_session_data_path):
  12. """
  13. Reads and processes operation data from multiple session data paths.
  15. This function loads operation data stored in 'ops.npy' files from a list of session
  16. directories, updates each operation dictionary with its corresponding session path,
  17. and returns a list of these operation dictionaries.
  19. Args:
  20. list_session_data_path (list): A list of strings, where each string is a path to
  21. a directory containing an 'ops.npy' file with operation data.
  23. Returns:
  24. list: A list of dictionaries, where each dictionary contains the operation data
  25. loaded from an 'ops.npy' file, with an additional 'save_path0' key pointing
  26. to the corresponding session directory.
  28. Notes:
  29. - The function uses `numpy.load` with `allow_pickle=True` to load the 'ops.npy'
  30. files, which are expected to contain pickled Python objects (dictionaries).
  31. - The 'ops.npy' file must exist in each session directory specified in
  32. `list_session_data_path`.
  33. - The function assumes that the loaded `ops` data is a dictionary and modifies it
  34. by adding or updating the 'save_path0' key with the session directory path.
  36. Example:
  37. >>> session_paths = ['/path/to/session1', '/path/to/session2']
  38. >>> ops_list = read_ops(session_paths)
  39. >>> print(ops_list[0]['save_path0'])
  40. '/path/to/session1'
  42. Raises:
  43. FileNotFoundError: If an 'ops.npy' file is not found in any of the specified
  44. session directories.
  45. ValueError: If the loaded 'ops.npy' file does not contain a dictionary.
  46. """
  47. list_ops = []
  48. for session_data_path in list_session_data_path:
  49. ops = np.load(
  50. os.path.join(session_data_path, 'ops.npy'),
  51. allow_pickle=True).item()
  52. ops['save_path0'] = os.path.join(session_data_path)
  53. list_ops.append(ops)
  54. return list_ops
  56. def create_memmap(data, dtype, mmap_path):
  57. """
  58. Creates a memory-mapped array from input data and saves it to a specified file path.
  60. This function creates a memory-mapped array with the same shape and data type as the
  61. input data, writes the input data to the memory-mapped file, and returns the
  62. memory-mapped array for further use.
  64. Args:
  65. data (numpy.ndarray): The input NumPy array whose data will be written to the
  66. memory-mapped file.
  67. dtype (numpy.dtype): The data type of the memory-mapped array.
  68. mmap_path (str): The file path where the memory-mapped array will be stored.
  70. Returns:
  71. numpy.memmap: A memory-mapped array with the specified shape and data type,
  72. containing the data from the input array.
  74. Notes:
  75. - The memory-mapped array is created in 'w+' mode, which creates a new file or
  76. overwrites an existing file for reading and writing.
  77. - The function uses `numpy.memmap` to create the memory-mapped array, which allows
  78. efficient handling of large arrays by mapping them directly to disk.
  79. - The input `data` is copied into the memory-mapped array using slice assignment.
  81. Example:
  82. >>> import numpy as np
  83. >>> data = np.array([[1, 2], [3, 4]], dtype=np.int32)
  84. >>> mmap_arr = create_memmap(data, np.int32, 'data.mmap')
  85. >>> print(mmap_arr)
  86. [[1 2]
  87. [3 4]]
  89. Raises:
  90. ValueError: If the `dtype` does not match the data type of the input `data` or if
  91. the `mmap_path` is invalid.
  92. OSError: If there are issues with file creation or access at `mmap_path`.
  93. """
  94. memmap_arr = np.memmap(mmap_path, dtype=dtype, mode='w+', shape=data.shape)
  95. memmap_arr[:] = data[...]
  96. return memmap_arr
  98. def get_memmap_path(ops, h5_file_name):
  99. """
  100. Generates paths for memory-mapped data and the corresponding HDF5 file.
  102. This function creates a directory for memory-mapped data based on the provided HDF5
  103. file name and constructs paths for both the memory-mapped directory and the HDF5 file.
  104. The memory-mapped directory is created under a 'memmap' subdirectory within the
  105. session's save path if it does not already exist.
  107. Args:
  108. ops (dict): A dictionary containing session metadata, including the key
  109. 'save_path0' which specifies the base directory path for the session.
  110. h5_file_name (str): The name of the HDF5 file (e.g., 'data.h5').
  112. Returns:
  113. tuple: A tuple containing two strings:
  114. - mm_path (str): The path to the memory-mapped data directory.
  115. - file_path (str): The path to the HDF5 file.
  117. Notes:
  118. - The memory-mapped directory is named after the HDF5 file name (without its
  119. extension) and is created under '<save_path0>/memmap/'.
  120. - If the memory-mapped directory does not exist, it is created automatically.
  121. - The function assumes that 'save_path0' exists in the `ops` dictionary and is a
  122. valid directory path.
  124. Example:
  125. >>> ops = {'save_path0': '/path/to/session'}
  126. >>> h5_file_name = 'data.h5'
  127. >>> mm_path, file_path = get_memmap_path(ops, h5_file_name)
  128. >>> print(mm_path)
  129. '/path/to/session/memmap/data'
  130. >>> print(file_path)
  131. '/path/to/session/data.h5'
  133. Raises:
  134. KeyError: If 'save_path0' is not present in the `ops` dictionary.
  135. OSError: If there are issues creating the memory-mapped directory or accessing
  136. the file system.
  137. """
  138. mm_folder_name, _ = os.path.splitext(h5_file_name)
  139. if not os.path.exists(os.path.join(ops['save_path0'], 'memmap', mm_folder_name)):
  140. os.makedirs(os.path.join(ops['save_path0'], 'memmap', mm_folder_name))
  141. mm_path = os.path.join(ops['save_path0'], 'memmap', mm_folder_name)
  142. file_path = os.path.join(ops['save_path0'], h5_file_name)
  143. return mm_path, file_path
  145. def read_masks(ops):
  146. """
  147. Reads mask-related data from an HDF5 file and creates memory-mapped arrays.
  149. This function retrieves paths for memory-mapped data and an HDF5 file using
  150. `get_memmap_path`, then loads specific datasets from the HDF5 file into memory-mapped
  151. arrays. It handles both functional and anatomical data, with anatomical data loaded
  152. only if the session has two channels.
  154. Args:
  155. ops (dict): A dictionary containing session metadata, including:
  156. - 'save_path0': The base directory path for the session.
  157. - 'nchannels': The number of channels in the session (e.g., 1 or 2).
  159. Returns:
  160. list: A list containing the following memory-mapped arrays (or None for anatomical
  161. data if `ops['nchannels'] != 2`):
  162. - labels (numpy.memmap): Array of integer labels (dtype: int8).
  163. - masks (numpy.memmap): Functional mask data (dtype: float32).
  164. - mean_func (numpy.memmap): Mean functional data (dtype: float32).
  165. - max_func (numpy.memmap): Maximum functional data (dtype: float32).
  166. - mean_anat (numpy.memmap or None): Mean anatomical data (dtype: float32) if
  167. `ops['nchannels'] == 2`, otherwise None.
  168. - masks_anat (numpy.memmap or None): Anatomical mask data (dtype: float32) if
  169. `ops['nchannels'] == 2`, otherwise None.
  171. Notes:
  172. - The function assumes the HDF5 file 'masks.h5' exists in the session directory
  173. specified by `ops['save_path0']` and contains the datasets 'labels',
  174. 'masks_func', 'mean_func', 'max_func', and optionally 'mean_anat' and
  175. 'masks_anat' for two-channel data.
  176. - Memory-mapped arrays are created using the `create_memmap` function and stored
  177. in a 'memmap' subdirectory under the session path.
  178. - The HDF5 file is accessed in read-only mode ('r') using `h5py.File`.
  180. Example:
  181. >>> ops = {'save_path0': '/path/to/session', 'nchannels': 2}
  182. >>> result = read_masks(ops)
  183. >>> print(result[0]) # Access the labels memory-mapped array
  184. <numpy.memmap object with dtype=int8>
  186. Raises:
  187. KeyError: If required keys ('save_path0', 'nchannels') are missing from `ops` or
  188. required datasets are missing from the HDF5 file.
  189. FileNotFoundError: If the 'masks.h5' file does not exist at the specified path.
  190. OSError: If there are issues creating memory-mapped files or accessing the file
  191. system.
  192. """
  193. mm_path, file_path = get_memmap_path(ops, 'masks.h5')
  194. with h5py.File(file_path, 'r') as f:
  195. labels = create_memmap(f['labels'], 'int8', os.path.join(mm_path, 'labels.mmap'))
  196. masks = create_memmap(f['masks_func'], 'float32', os.path.join(mm_path, 'masks_func.mmap'))
  197. mean_func = create_memmap(f['mean_func'], 'float32', os.path.join(mm_path, 'mean_func.mmap'))
  198. max_func = create_memmap(f['max_func'], 'float32', os.path.join(mm_path, 'max_func.mmap'))
  199. mean_anat = create_memmap(f['mean_anat'], 'float32', os.path.join(mm_path, 'mean_anat.mmap')) if ops['nchannels'] == 2 else None
  200. masks_anat = create_memmap(f['masks_anat'], 'float32', os.path.join(mm_path, 'masks_anat.mmap')) if ops['nchannels'] == 2 else None
  201. return [labels, masks, mean_func, max_func, mean_anat, masks_anat]
  204. def read_raw_voltages(ops):
  205. """
  206. Reads raw voltage data from an HDF5 file and creates memory-mapped arrays.
  208. This function retrieves paths for memory-mapped data and an HDF5 file using
  209. `get_memmap_path`, then loads specific voltage-related datasets from the HDF5 file
  210. into memory-mapped arrays. The datasets include timestamps and various voltage signals
  211. related to experimental events (e.g., visual stimuli, audio stimuli, imaging triggers).
  213. Args:
  214. ops (dict): A dictionary containing session metadata, including:
  215. - 'save_path0': The base directory path for the session.
  217. Returns:
  218. list: A list containing the following memory-mapped arrays:
  219. - vol_time (numpy.memmap): Timestamps for voltage signals (dtype: float32).
  220. - vol_start (numpy.memmap): Trial start trigger signals (dtype: int8).
  221. - vol_stim_vis (numpy.memmap): Visual stimulus signals (dtype: int8).
  222. - vol_img (numpy.memmap): Imaging trigger signals (dtype: int8).
  223. - vol_hifi (numpy.memmap): HiFi (audio) trigger signals (dtype: int8).
  224. - vol_stim_aud (numpy.memmap): Audio stimulus signals (dtype: float32).
  225. - vol_flir (numpy.memmap): FLIR camera trigger signals (dtype: int8).
  226. - vol_pmt (numpy.memmap): PMT (photomultiplier tube) signals (dtype: int8).
  227. - vol_led (numpy.memmap): LED trigger signals (dtype: int8).
  229. Notes:
  230. - The function assumes the HDF5 file 'raw_voltages.h5' exists in the session
  231. directory specified by `ops['save_path0']` and contains a 'raw' group with
  232. datasets: 'vol_time', 'vol_start', 'vol_stim_vis', 'vol_img', 'vol_hifi',
  233. 'vol_stim_aud', 'vol_flir', 'vol_pmt', and 'vol_led'.
  234. - Memory-mapped arrays are created using the `create_memmap` function and stored
  235. in a 'memmap' subdirectory under the session path.
  236. - The HDF5 file is accessed in read-only mode ('r') using `h5py.File`.
  238. Example:
  239. >>> ops = {'save_path0': '/path/to/session'}
  240. >>> voltages = read_raw_voltages(ops)
  241. >>> print(voltages[0]) # Access the vol_time memory-mapped array
  242. <numpy.memmap object with dtype=float32>
  244. Raises:
  245. KeyError: If required keys (e.g., 'save_path0') are missing from `ops` or if
  246. expected datasets are missing in the HDF5 file.
  247. FileNotFoundError: If the 'raw_voltages.h5' file does not exist at the specified
  248. path.
  249. OSError: If there are issues creating memory-mapped files or accessing the file
  250. system.
  251. """
  252. mm_path, file_path = get_memmap_path(ops, 'raw_voltages.h5')
  253. with h5py.File(file_path, 'r') as f:
  254. vol_time = create_memmap(f['raw']['vol_time'], 'float32', os.path.join(mm_path, 'vol_time.mmap'))
  255. vol_start = create_memmap(f['raw']['vol_start'], 'int8', os.path.join(mm_path, 'vol_start.mmap'))
  256. vol_stim_vis = create_memmap(f['raw']['vol_stim_vis'], 'int8', os.path.join(mm_path, 'vol_stim_vis.mmap'))
  257. vol_hifi = create_memmap(f['raw']['vol_hifi'], 'int8', os.path.join(mm_path, 'vol_hifi.mmap'))
  258. vol_img = create_memmap(f['raw']['vol_img'], 'int8', os.path.join(mm_path, 'vol_img.mmap'))
  259. vol_stim_aud = create_memmap(f['raw']['vol_stim_aud'], 'float32', os.path.join(mm_path, 'vol_stim_aud.mmap'))
  260. vol_flir = create_memmap(f['raw']['vol_flir'], 'int8', os.path.join(mm_path, 'vol_flir.mmap'))
  261. vol_pmt = create_memmap(f['raw']['vol_pmt'], 'int8', os.path.join(mm_path, 'vol_pmt.mmap'))
  262. vol_led = create_memmap(f['raw']['vol_led'], 'int8', os.path.join(mm_path, 'vol_led.mmap'))
  263. return [vol_time, vol_start, vol_stim_vis, vol_img,
  264. vol_hifi, vol_stim_aud, vol_flir,
  265. vol_pmt, vol_led]
  267. def read_dff(ops):
  268. """
  269. Reads dF/F (delta F over F) traces from an HDF5 file and creates a memory-mapped array.
  271. This function retrieves paths for memory-mapped data and an HDF5 file using
  272. `get_memmap_path`, then loads the dF/F dataset from the HDF5 file into a
  273. memory-mapped array.
  275. Args:
  276. ops (dict): A dictionary containing session metadata, including:
  277. - 'save_path0': The base directory path for the session.
  279. Returns:
  280. numpy.memmap: A memory-mapped array containing dF/F traces (dtype: float32).
  282. Notes:
  283. - The function assumes the HDF5 file 'dff.h5' exists in the session directory
  284. specified by `ops['save_path0']` and contains a 'dff' dataset.
  285. - The memory-mapped array is created using the `create_memmap` function and
  286. stored in a 'memmap' subdirectory under the session path.
  287. - The HDF5 file is accessed in read-only mode ('r') using `h5py.File`.
  289. Example:
  290. >>> ops = {'save_path0': '/path/to/session'}
  291. >>> dff_data = read_dff(ops)
  292. >>> print(dff_data) # Access the dF/F memory-mapped array
  293. <numpy.memmap object with dtype=float32>
  295. Raises:
  296. KeyError: If required keys (e.g., 'save_path0') are missing from `ops` or if the
  297. 'dff' dataset is missing in the HDF5 file.
  298. FileNotFoundError: If the 'dff.h5' file does not exist at the specified path.
  299. OSError: If there are issues creating the memory-mapped file or accessing the file
  300. system.
  301. """
  302. mm_path, file_path = get_memmap_path(ops, 'dff.h5')
  303. with h5py.File(file_path, 'r') as f:
  304. dff = create_memmap(f['dff'], 'float32', os.path.join(mm_path, 'dff.mmap'))
  305. return dff
  307. # Reading bpod from matlab file
  308. def read_bpod_mat_data(ops, session_start_time):
  309. """
  310. Reads and processes Bpod session data from a MATLAB file into a structured DataFrame.
  312. This function loads behavioral data from a 'bpod_session_data.mat' file, processes
  313. trial-related information (e.g., trial timings, outcomes, stimulus sequences, and
  314. licking events), and organizes it into a pandas DataFrame. It handles nested MATLAB
  315. structures and converts them into Python dictionaries, adjusts timestamps relative to
  316. a session start time, and labels trial outcomes and states.
  318. Args:
  319. ops (dict): A dictionary containing session metadata, including:
  320. - 'save_path0': The base directory path where the 'bpod_session_data.mat' file
  321. is located.
  322. session_start_time (float): The reference start time (in milliseconds) for
  323. correcting trial timestamps.
  325. Returns:
  326. pandas.DataFrame: A DataFrame containing trial-related data with the following
  327. columns:
  328. - time_trial_start (float): Trial start timestamps (ms, adjusted to session
  329. start).
  330. - time_trial_end (float): Trial end timestamps (ms, adjusted to session start).
  331. - trial_type (int): Trial type indicator (0-based, derived from raw data).
  332. - outcome (str): Trial outcome ('punish', 'reward', 'naive_punish',
  333. 'naive_reward', 'no_choose', or 'other').
  334. - state_window_choice (array): Timestamps for the 'WindowChoice' state [start, end].
  335. - state_reward (array): Timestamps for the 'Reward' state [start, end].
  336. - state_punish (array): Timestamps for the 'Punish' state [start, end].
  337. - stim_seq (array): Stimulus sequence timestamps [[BNC1High], [BNC1Low]].
  338. - isi (float): Inter-stimulus interval (ms) between BNC1High and BNC1Low.
  339. - lick (array): Licking events with [timestamps, direction, correctness, lick_type].
  341. Notes:
  342. - The function assumes the 'bpod_session_data.mat' file exists in the directory
  343. specified by `ops['save_path0']` and contains a 'SessionData' structure with
  344. fields like 'nTrials', 'RawEvents', 'TrialStartTimestamp', 'TrialEndTimestamp',
  345. and 'TrialTypes'.
  346. - Timestamps are converted to milliseconds and adjusted relative to the session
  347. start time.
  348. - Nested MATLAB structures are recursively converted to Python dictionaries using
  349. helper functions `_check_keys`, `_todict`, and `_tolist`.
  350. - Trial outcomes are determined by the `states_labeling` helper function based on
  351. the presence of specific states in the trial data.
  352. - Stimulus sequences and licking events are processed to handle missing or invalid
  353. data (e.g., NaN values for absent events).
  354. - The 'yicong_forever' string is used as a temporary placeholder to handle array
  355. length alignment and is removed before returning the DataFrame.
  357. Example:
  358. >>> ops = {'save_path0': '/path/to/session'}
  359. >>> session_start_time = 1000.0
  360. >>> df = read_bpod_mat_data(ops, session_start_time)
  361. >>> print(df[['time_trial_start', 'outcome']].head())
  362. time_trial_start outcome
  363. 0 1000.0 reward
  364. 1 1500.0 punish
  365. ...
  367. Raises:
  368. KeyError: If required keys (e.g., 'save_path0') are missing from `ops` or if
  369. expected fields are missing in the MATLAB file.
  370. FileNotFoundError: If the 'bpod_session_data.mat' file does not exist.
  371. ValueError: If the data structure in the MATLAB file is malformed or incompatible.
  372. """
  373. def _check_keys(d):
  374. """
  375. Recursively converts MATLAB structs to dictionaries.
  377. Args:
  378. d (dict): Input dictionary containing MATLAB struct objects.
  380. Returns:
  381. dict: Dictionary with MATLAB structs converted to nested dictionaries.
  382. """
  383. for key in d:
  384. if isinstance(d[key], sio.matlab.mat_struct):
  385. d[key] = _todict(d[key])
  386. return d
  388. def _todict(matobj):
  389. """
  390. Converts a MATLAB struct to a Python dictionary.
  392. Args:
  393. matobj (sio.matlab.mat_struct): MATLAB struct object.
  395. Returns:
  396. dict: Dictionary with field names as keys and converted values.
  397. """
  398. d = {}
  399. for strg in matobj._fieldnames:
  400. elem = matobj.__dict__[strg]
  401. if isinstance(elem, sio.matlab.mat_struct):
  402. d[strg] = _todict(elem)
  403. elif isinstance(elem, np.ndarray):
  404. d[strg] = _tolist(elem)
  405. else:
  406. d[strg] = elem
  407. return d
  409. def _tolist(ndarray):
  410. """
  411. Recursively converts NumPy arrays to lists, handling nested MATLAB structs.
  413. Args:
  414. ndarray (numpy.ndarray): Input NumPy array.
  416. Returns:
  417. list: List of converted elements.
  418. """
  419. elem_list = []
  420. for sub_elem in ndarray:
  421. if isinstance(sub_elem, sio.matlab.mat_struct):
  422. elem_list.append(_todict(sub_elem))
  423. elif isinstance(sub_elem, np.ndarray):
  424. elem_list.append(_tolist(sub_elem))
  425. else:
  426. elem_list.append(sub_elem)
  427. return elem_list
  429. def states_labeling(trial_states):
  430. """
  431. Labels trial outcomes based on trial states.
  433. Args:
  434. trial_states (dict): Dictionary of trial states and their timestamps.
  436. Returns:
  437. str: Outcome label ('punish', 'reward', 'naive_punish', 'naive_reward',
  438. 'no_choose', or 'other').
  439. """
  440. if 'Punish' in trial_states.keys() and not np.isnan(trial_states['Punish'][0]):
  441. outcome = 'punish'
  442. elif 'Reward' in trial_states.keys() and not np.isnan(trial_states['Reward'][0]):
  443. outcome = 'reward'
  444. elif 'PunishNaive' in trial_states.keys() and not np.isnan(trial_states['PunishNaive'][0]):
  445. outcome = 'naive_punish'
  446. elif 'RewardNaive' in trial_states.keys() and not np.isnan(trial_states['RewardNaive'][0]):
  447. outcome = 'naive_reward'
  448. elif 'DidNotChoose' in trial_states.keys() and not np.isnan(trial_states['DidNotChoose'][0]):
  449. outcome = 'no_choose'
  450. else:
  451. outcome = 'other'
  452. return outcome
  454. def get_state(trial_state_dict, target_state, trial_start):
  455. """
  456. Retrieves timestamps for a specific trial state.
  458. Args:
  459. trial_state_dict (dict): Dictionary of trial states.
  460. target_state (str): Name of the target state.
  461. trial_start (float): Trial start timestamp (ms).
  463. Returns:
  464. numpy.ndarray: Array of [start, end] timestamps (ms) or [NaN, NaN] if the
  465. state is not found.
  466. """
  467. if target_state in trial_state_dict:
  468. time_state = 1000 * np.array(trial_state_dict[target_state]) + trial_start
  469. else:
  470. time_state = np.array([np.nan, np.nan])
  471. return time_state
  473. # Read raw data from MATLAB file
  474. raw = sio.loadmat(
  475. os.path.join(ops['save_path0'], 'bpod_session_data.mat'),
  476. struct_as_record=False, squeeze_me=True)
  477. raw = _check_keys(raw)['SessionData']
  479. trial_labels = dict()
  480. n_trials = raw['nTrials']
  481. trial_states = [raw['RawEvents']['Trial'][ti]['States'] for ti in range(n_trials)]
  482. trial_events = [raw['RawEvents']['Trial'][ti]['Events'] for ti in range(n_trials)]
  484. # Trial start and end timestamps
  485. trial_labels['time_trial_start'] = 1000 * np.array(raw['TrialStartTimestamp']).reshape(-1)
  486. trial_labels['time_trial_end'] = 1000 * np.array(raw['TrialEndTimestamp']).reshape(-1)
  487. trial_labels['time_trial_end'] = trial_labels['time_trial_end'] - trial_labels['time_trial_start'][0] + session_start_time
  488. trial_labels['time_trial_start'] = trial_labels['time_trial_start'] - trial_labels['time_trial_start'][0] + session_start_time
  490. # Trial type Block type and outcome
  491. trial_labels['trial_type'] = np.array(raw['TrialTypes']).reshape(-1) - 1
  492. trial_labels['block_type'] = np.array(raw['BlockTypes']).reshape(-1)
  493. trial_labels['outcome'] = np.array([states_labeling(ts) for ts in trial_states], dtype='object')
  495. # Trial state timings
  496. trial_labels['state_window_choice'] = np.array([
  497. get_state(trial_states[ti], 'WindowChoice', trial_labels['time_trial_start'][ti])
  498. for ti in range(n_trials)] + ['yicong_forever'], dtype='object')[:-1]
  499. trial_labels['state_reward'] = np.array([
  500. get_state(trial_states[ti], 'Reward', trial_labels['time_trial_start'][ti])
  501. for ti in range(n_trials)] + ['yicong_forever'], dtype='object')[:-1]
  502. trial_labels['state_punish'] = np.array([
  503. get_state(trial_states[ti], 'Punish', trial_labels['time_trial_start'][ti])
  504. for ti in range(n_trials)] + ['yicong_forever'], dtype='object')[:-1]
  506. # Stimulus timing
  507. trial_isi = []
  508. trial_stim_seq = []
  509. for ti in range(n_trials):
  510. if ('BNC1High' in trial_events[ti].keys() and
  511. 'BNC1Low' in trial_events[ti].keys() and
  512. len(np.array(trial_events[ti]['BNC1High']).reshape(-1)) == 2 and
  513. len(np.array(trial_events[ti]['BNC1Low']).reshape(-1)) == 2):
  514. stim_seq = 1000 * np.array([trial_events[ti]['BNC1High'], trial_events[ti]['BNC1Low']]) + trial_labels['time_trial_start'][ti]
  515. stim_seq = np.transpose(stim_seq, [1, 0])
  516. isi = 1000 * np.array(trial_events[ti]['BNC1High'][1] - trial_events[ti]['BNC1Low'][0])
  517. else:
  518. stim_seq = np.array([[np.nan, np.nan], [np.nan, np.nan]])
  519. isi = np.nan
  520. trial_stim_seq.append(stim_seq)
  521. trial_isi.append(isi)
  522. trial_labels['stim_seq'] = np.array(trial_stim_seq + ['yicong_forever'], dtype='object')[:-1]
  523. trial_labels['isi'] = np.array(trial_isi + ['yicong_forever'], dtype='object')[:-1]
  525. # Licking events
  526. trial_lick = []
  527. for ti in range(n_trials):
  528. licking_events = []
  529. direction = []
  530. correctness = []
  531. if 'Port1In' in trial_events[ti].keys():
  532. lick_left = np.array(trial_events[ti]['Port1In']).reshape(-1)
  533. licking_events.append(lick_left)
  534. direction.append(np.zeros_like(lick_left))
  535. if trial_labels['trial_type'][ti] == 0:
  536. correctness.append(np.ones_like(lick_left))
  537. else:
  538. correctness.append(np.zeros_like(lick_left))
  539. if 'Port3In' in trial_events[ti].keys():
  540. lick_right = np.array(trial_events[ti]['Port3In']).reshape(-1)
  541. licking_events.append(lick_right)
  542. direction.append(np.ones_like(lick_right))
  543. if trial_labels['trial_type'][ti] == 1:
  544. correctness.append(np.ones_like(lick_right))
  545. else:
  546. correctness.append(np.zeros_like(lick_right))
  547. if len(licking_events) > 0:
  548. licking_events = 1000 * np.concatenate(licking_events).reshape(1, -1) + trial_labels['time_trial_start'][ti]
  549. direction = np.concatenate(direction).reshape(1, -1)
  550. correctness = np.concatenate(correctness).reshape(1, -1)
  551. lick = np.concatenate([licking_events, direction, correctness], axis=0)
  552. lick = lick[:, np.argsort(lick[0, :])]
  553. lick = lick[:, lick[0, :] >= trial_labels['state_window_choice'][ti][0]]
  554. if np.size(lick) != 0:
  555. lick_type = np.full(lick.shape[1], np.nan)
  556. lick_type[0] = 1
  557. if (not np.isnan(trial_labels['state_reward'][ti][1]) and
  558. len(lick_type) > 1):
  559. lick_type[1:][lick[0, 1:] > trial_labels['state_reward'][ti][0]] = 0
  560. lick_type = lick_type.reshape(1, -1)
  561. lick = np.concatenate([lick, lick_type], axis=0)
  562. else:
  563. lick = np.array([[np.nan], [np.nan], [np.nan], [np.nan]])
  564. else:
  565. lick = np.array([[np.nan], [np.nan], [np.nan], [np.nan]])
  566. trial_lick.append(lick)
  567. trial_labels['lick'] = np.array(trial_lick + ['yicong_forever'], dtype='object')[:-1]
  569. # Convert to DataFrame
  570. trial_labels = pd.DataFrame(trial_labels)
  571. return trial_labels
  573. # Readiing Trialized data
  574. def read_trial_label(ops):
  575. """Read trial label data from a CSV file into a pandas DataFrame.
  577. This function reads a CSV file containing trial label data and converts specific columns
  578. into numpy arrays with appropriate data types and shapes. Some columns containing array-like
  579. data are parsed from string representations back into numpy arrays. The processed data is
  580. returned as a structured pandas DataFrame.
  582. Args:
  583. ops (dict): Dictionary containing configuration options, including 'save_path0' for the
  584. directory where the 'trial_labels.csv' file is located.
  586. Returns:
  587. pandas.DataFrame: A DataFrame containing the trial label data with columns:
  588. - time_trial_start (float32): Start times of trials.
  589. - time_trial_end (float32): End times of trials.
  590. - trial_type (int8): Type of each trial.
  591. - outcome (object): Outcome of each trial.
  592. - state_window_choice (object): Array of choice window states per trial.
  593. - state_reward (object): Array of reward states per trial.
  594. - state_punish (object): Array of punishment states per trial.
  595. - stim_seq (object): 2D array of stimulus sequences per trial.
  596. - isi (float32): Inter-stimulus intervals.
  597. - lick (object): 2D array of lick data per trial.
  599. Notes:
  600. - The CSV file is expected to be located at ops['save_path0']/trial_labels.csv.
  601. - Columns with array-like data (e.g., state_window_choice, stim_seq, lick) are stored as
  602. strings in the CSV and are parsed back into numpy arrays with specified shapes.
  603. - The function uses a helper function, object_parse, to handle array parsing.
  604. - The 'yicong_forever' string is appended during parsing and then removed to maintain
  605. data integrity.
  606. """
  607. raw_csv = pd.read_csv(os.path.join(ops['save_path0'], 'trial_labels.csv'), index_col=0)
  608. # recover object numpy array from csv str.
  609. def object_parse(k, shape):
  610. arr = np.array(
  611. [np.fromstring(s.replace('[', '').replace(']', ''), sep=' ').reshape(shape)
  612. for s in raw_csv[k].to_list()] + ['yicong_forever'],
  613. dtype='object')[:-1]
  614. return arr
  615. # parse all array.
  616. time_trial_start = raw_csv['time_trial_start'].to_numpy(dtype='float32')
  617. time_trial_end = raw_csv['time_trial_end'].to_numpy(dtype='float32')
  618. trial_type = raw_csv['trial_type'].to_numpy(dtype='int8')
  619. block_type = raw_csv['block_type'].to_numpy(dtype='int8')
  620. outcome = raw_csv['outcome'].to_numpy(dtype='object')
  621. state_window_choice = object_parse('state_window_choice', [-1])
  622. state_reward = object_parse('state_reward', [-1])
  623. state_punish = object_parse('state_punish', [-1])
  624. stim_seq = object_parse('stim_seq', [-1, 2])
  625. isi = raw_csv['isi'].to_numpy(dtype='float32')
  626. lick = object_parse('lick', [4, -1])
  627. # convert to dataframe.
  628. trial_labels = pd.DataFrame({
  629. 'time_trial_start': time_trial_start,
  630. 'time_trial_end': time_trial_end,
  631. 'trial_type': trial_type,
  632. 'block_type': block_type,
  633. 'outcome': outcome,
  634. 'state_window_choice': state_window_choice,
  635. 'state_reward': state_reward,
  636. 'state_punish': state_punish,
  637. 'stim_seq': stim_seq,
  638. 'isi': isi,
  639. 'lick': lick,
  640. })
  641. return trial_labels
  643. def read_neural_trials(ops, smooth):
  644. """Read neural trial data from an HDF5 file and create memory-mapped arrays.
  646. This function reads neural trial data from an HDF5 file, including fluorescence signals,
  647. time points, and voltage signals, and optionally applies a Savitzky-Golay filter to smooth
  648. the fluorescence data. The data is stored as memory-mapped arrays for efficient access and
  649. returned as a dictionary. Trial labels are read using the read_trial_label function.
  651. Args:
  652. ops (dict): Dictionary containing configuration options, including 'save_path0' for the
  653. directory where the 'neural_trials.h5' file is located.
  654. smooth (bool): If True, apply Savitzky-Golay smoothing to the fluorescence data (dff).
  656. Returns:
  657. dict: A dictionary containing memory-mapped arrays and trial labels:
  658. - dff (numpy.memmap): Fluorescence signal data (float32).
  659. - time (numpy.memmap): Corrected neural time points (float32).
  660. - trial_labels (pandas.DataFrame): Trial label data from read_trial_label.
  661. - vol_time (numpy.memmap): Voltage signal time points (float32).
  662. - vol_stim_vis (numpy.memmap): Visual stimulation signals (int8).
  663. - vol_stim_aud (numpy.memmap): Auditory stimulation signals (float32).
  664. - vol_flir (numpy.memmap): FLIR camera signals (int8).
  665. - vol_pmt (numpy.memmap): Photomultiplier tube signals (int8).
  666. - vol_led (numpy.memmap): LED signals (int8).
  668. Notes:
  669. - The HDF5 file is expected to be located at ops['save_path0']/neural_trials.h5.
  670. - If smooth is True, a Savitzky-Golay filter is applied to the dff data with a window
  671. length of 9 and polynomial order of 3.
  672. - Memory-mapped files are created in a directory specified by get_memmap_path.
  673. - The function assumes the existence of helper functions: get_memmap_path,
  674. read_trial_label, and create_memmap.
  675. """
  676. mm_path, file_path = get_memmap_path(ops, 'neural_trials.h5')
  677. trial_labels = read_trial_label(ops)
  678. with h5py.File(file_path, 'r') as f:
  679. neural_trials = dict()
  680. dff = np.array(f['neural_trials']['dff'])
  681. if smooth:
  682. window_length = 9
  683. polyorder = 3
  684. dff = np.apply_along_axis(
  685. savgol_filter, 1, dff,
  686. window_length=window_length,
  687. polyorder=polyorder)
  688. else:
  689. pass
  690. neural_trials['dff'] = create_memmap(dff, 'float32', os.path.join(mm_path, 'dff.mmap'))
  691. neural_trials['time'] = create_memmap(f['neural_trials']['time'], 'float32', os.path.join(mm_path, 'time.mmap'))
  692. neural_trials['trial_labels'] = trial_labels
  693. neural_trials['vol_time'] = create_memmap(f['neural_trials']['vol_time'], 'float32', os.path.join(mm_path, 'vol_time.mmap'))
  694. neural_trials['vol_stim_vis'] = create_memmap(f['neural_trials']['vol_stim_vis'], 'int8', os.path.join(mm_path, 'vol_stim_vis.mmap'))
  695. neural_trials['vol_stim_aud'] = create_memmap(f['neural_trials']['vol_stim_aud'], 'float32', os.path.join(mm_path, 'vol_stim_aud.mmap'))
  696. neural_trials['vol_flir'] = create_memmap(f['neural_trials']['vol_flir'], 'int8', os.path.join(mm_path, 'vol_flir.mmap'))
  697. neural_trials['vol_pmt'] = create_memmap(f['neural_trials']['vol_pmt'], 'int8', os.path.join(mm_path, 'vol_pmt.mmap'))
  698. neural_trials['vol_led'] = create_memmap(f['neural_trials']['vol_led'], 'int8', os.path.join(mm_path, 'vol_led.mmap'))
  699. return neural_trials
  701. # clean memory mapping files.
  702. def clean_memap_path(ops):
  703. try:
  704. if os.path.exists(os.path.join(ops['save_path0'], 'memmap')):
  705. shutil.rmtree(os.path.join(ops['save_path0'], 'memmap'))
  706. except: pass

reader.py at commit 1c0f5de, no license · at the source

Overview

Authors: Yicong Huang1, Ali Shamsnia2, Mengze Chen2, Shuang Wu2, Timothy Stamm2, Sophie Medico2, Farzaneh Najafi2
ORCID iDs: Yicong Huang, Ali Shamsnia, Mengze Chen, Shuang Wu, Timothy Stamm, Sophie Medico, Farzaneh Najafi
  1. Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA 30332, USA
  2. School of Biological Sciences, Georgia Institute of Technology, Atlanta, GA, 30332, USA
Institutions: Georgia Institute of Technology (United States); Emory University (United States); The Wallace H. Coulter Department of Biomedical Engineering (United States)
Journal: Science advances, volume 12, issue 34, article eaed6417
Dates: received 4 November 2025; accepted 14 July 2026; published online 21 August 2026; in print August 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1126/sciadv.aed6417 · PMID 42627925 · PMCID PMC13496207 · OpenAlex W7203947945
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: mouse (organism), systems (subfield)
Methods: Smoothing, state filtering, decompositions, Machine learning, Statistics, Preprocessing, Connectivity, fMRI & imaging, Single-unit activity, calcium imaging
MeSH: Parietal Lobe*, Visual Cortex*, Animals, Mice, Neurons, Photic Stimulation, Time Factors (* major topic)
Journal subjects: Neuroscience
Topic: Neural dynamics and brain function (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: Research Corporation for Science Advancement (AWD-007029, AWD-007028); Whitehall Foundation (Whitehall Foundation, Inc.) (AWD-006942); Georgia Institute of Technology (Start Up Funds); Chan Zuckerberg Initiative (AWD-005936, AWD-006703)
Citations: not cited yet (Europe PMC); 135 references in the paper

Abstract

Ramping neural activity is widely interpreted as a signature of predictive processing, but whether these signals truly reflect predictions or instead emerge from sensory mechanisms remains unclear. To address this question, we used two-photon calcium imaging across multiple cell types in visual and parietal cortex, while awake mice passively received repeated audiovisual stimuli presented under distinct temporal structures. Neurons segregated into two broad response classes: stimulus-activated (ramp-down) and stimulus-inhibited (ramp-up) populations with diverse temporal kinetics. Multiple findings argued against a predictive interpretation: Ramping activity was present in naïve animals, neural responses changed immediately after short/long interval transitions, and unexpected stimulus timings elicited nearly identical responses in predictable and irregular contexts. Population analyses further showed that ramping reflected relaxation from stimulus-evoked activity rather than anticipatory buildup. Heterogeneous kinetics generated a robust population code for elapsed time. Together, these findings show that neural ramps during passive stimulation arise from stimulus-evoked dynamics that intrinsically generate temporal signals, rather than from temporal predictive processing.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 2 matches between paragraphs and lines of code.

najafi-laboratory/2p_imaging

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 1c0f5de0cc6b9e430169e1a5a27f643dac8f7114, 16 September 2026
Languages: Python (372), Jupyter (26), Shell (17)
Size: 681 files, 415 scripts
Software Heritage: not archived
Found in: “Data, code, and materials availability:”
Holds: README, environment (environment-preprocessing-qc.yml, pyproject.toml, requirements-docs.txt, tracking/environment.yml, utils_2p/environment-preprocessing-qc-suite2p-0x.yml, utils_2p/environment-preprocessing-qc-suite2p-1x.yml, utils_2p/environment-processing-suite2p-0x.yml, utils_2p/environment-processing-suite2p-1x.yml, visualize_2p_video_202412/requirements.txt), tests, continuous integration, documentation, 26 notebooks
Not found: license file, CITATION.cff
Tools: NumPy (57 files), Matplotlib (41 files), SciPy (25 files), pandas (23 files), h5py (20 files), scikit-learn (16 files), seaborn (16 files), scikit-image (6 files), Plotly (2 files)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
63 files
At the source: github.com/najafi-laboratory/2p_imaging/tree/main/passive_interval_oddball_202412

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 63 scripts, each with its path and the digest of its content;
  • 2 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data, code, and materials availability

All data and code needed to evaluate and reproduce the results in this paper are present in the paper and/or the Supplementary Materials. This study did not generate new materials. Data used in this paper are available at Zenodo: https://doi.org/10.5281/zenodo.20350578. Code for data processing and analysis is publicly available on the Najafi Lab GitHub page: https://github.com/najafi-laboratory/2p_imaging/tree/main/passive_interval_oddball_202412.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 7 authors, 7 MeSH terms, 4 funders, 133 references.

Cite

This paper

Huang, Y., Shamsnia, A., Chen, M., Wu, S., Stamm, T., Medico, S., & Najafi, F. (2026). Intrinsic timing, not temporal prediction, underlies ramping dynamics in visual and parietal cortex during passive behavior. Science advances, 12(34), eaed6417. https://doi.org/10.1126/sciadv.aed6417

BibTeX

@article{huang2026intrinsic,
author = {Huang, Yicong and Shamsnia, Ali and Chen, Mengze and Wu, Shuang and Stamm, Timothy and Medico, Sophie and Najafi, Farzaneh},
title = {{Intrinsic timing, not temporal prediction, underlies ramping dynamics in visual and parietal cortex during passive behavior}},
journal = {Science advances},
year = {2026},
month = aug,
volume = {12},
number = {34},
pages = {eaed6417},
publisher = {American Association for the Advancement of Science},
issn = {2375-2548},
doi = {10.1126/sciadv.aed6417},
url = {https://doi.org/10.1126/sciadv.aed6417},
pmid = {42627925},
pmcid = {PMC13496207}
}

RIS

TY - JOUR
AU - Huang, Yicong
AU - Shamsnia, Ali
AU - Chen, Mengze
AU - Wu, Shuang
AU - Stamm, Timothy
AU - Medico, Sophie
AU - Najafi, Farzaneh
TI - Intrinsic timing, not temporal prediction, underlies ramping dynamics in visual and parietal cortex during passive behavior
T2 - Science advances
J2 - Sci Adv
PY - 2026
DA - 2026/08/21
VL - 12
IS - 34
SP - eaed6417
SN - 2375-2548
PB - American Association for the Advancement of Science
DO - 10.1126/sciadv.aed6417
UR - https://doi.org/10.1126/sciadv.aed6417
LA - en
ER -

CSL-JSON

{
"id": "10.1126/sciadv.aed6417",
"type": "article-journal",
"title": "Intrinsic timing, not temporal prediction, underlies ramping dynamics in visual and parietal cortex during passive behavior",
"container-title": "Science advances",
"author": [
{
"family": "Huang",
"given": "Yicong"
},
{
"family": "Shamsnia",
"given": "Ali"
},
{
"family": "Chen",
"given": "Mengze"
},
{
"family": "Wu",
"given": "Shuang"
},
{
"family": "Stamm",
"given": "Timothy"
},
{
"family": "Medico",
"given": "Sophie"
},
{
"family": "Najafi",
"given": "Farzaneh"
}
],
"container-title-short": "Sci Adv",
"volume": "12",
"issue": "34",
"page": "eaed6417",
"DOI": "10.1126/sciadv.aed6417",
"PMID": "42627925",
"PMCID": "PMC13496207",
"ISSN": "2375-2548",
"publisher": "American Association for the Advancement of Science",
"URL": "https://doi.org/10.1126/sciadv.aed6417",
"language": "en",
"issued": {
"date-parts": [
[
2026,
8,
21
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-73999-w [code]
Independence and coherence in temporal sequence computation across the fronto-parietal network.
Journal: Nature communications
In common: scikit-learn, SciPy, Matplotlib, 1 other tool, systems, mouse, 10 references
[2] doi:10.1038/s41593-026-02255-7 [code]
Neural circuits encode prior knowledge of temporal statistics.
Journal: Nature neuroscience
In common: Plotly, scikit-learn, pandas, 3 other tools, systems, mouse, 5 references
[3] doi:10.1038/s41586-026-10528-1 [code]
A critical initialization for biological neural networks.
Journal: Nature
In common: scikit-learn, SciPy, Matplotlib, 1 other tool, mouse, 8 references
[4] doi:10.1016/j.neuron.2026.07.016 [code]
Inferring brain-wide interactions using data-constrained recurrent neural network models.
Journal: Neuron
In common: Matplotlib, NumPy, systems, mouse, 8 references
[5] doi:10.1038/s41593-026-02350-9 [code]
Probing inter-areal computations with a two-photon holographic mesoscope.
Journal: Nature neuroscience
In common: h5py, SciPy, NumPy, systems, mouse, 6 references
[6] doi:10.1371/journal.pbio.3003915 [code]
Noise-invariant representations of sound emerge along the canonical cortical hierarchy.
Journal: PLoS biology
In common: scikit-image, h5py, seaborn, 5 other tools, mouse, 3 references
[7] doi:10.1371/journal.pone.0356243 [code]
Functional organization and natural scene responses across mouse visual cortical areas revealed with encoding manifolds.
Journal: PloS one
In common: Plotly, scikit-learn, pandas, 3 other tools, mouse, 4 references
[8] doi:10.1371/journal.pbio.3003831 [code]
Disinhibitory signaling enables flexible coding of top-down information in cortical networks.
Journal: PLoS biology
In common: h5py, scikit-learn, pandas, 3 other tools, systems, mouse, 4 references
[9] doi:10.1038/s41467-026-71725-0 [code]
Interactions across hemispheres in prefrontal cortex reflect global cognitive processing.
Journal: Nature communications
In common: scikit-learn, pandas, SciPy, 2 other tools, 6 references
[10] doi:10.1523/jneurosci.0987-25.2026 [code]
Cell-Type-Specific Synaptic Scaling Mechanisms Differentially Contribute to Associative Learning.
Journal: The Journal of neuroscience : the official journal of the Society for Neuroscience
In common: seaborn, pandas, SciPy, 2 other tools, 5 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.