OSCR

Movie reconstruction from mouse visual cortex activity.

Code ↔ Paper

12 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 12 matches
  1. [1] § Methods › State-of-the-art dynamic neural encoding model ↔ src/models/dwiseneuro.py, lines 343–405 · score 0.70 · SiLU, DwiseNeuro, Softplus, readout, kernel, cortex
  2. [2] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › Not all spatial and temporal frequencies are reconstructed equally ↔ scripts/analyse_reconstructions_gaussian_noise.ipynb, lines 11–81 · score 0.61 · phase inverted, Reconstructed Gaussian, Gaussian noise, temporal length constants, stimulus, mask
  3. [3] § Methods › Additional visual stimuli ↔ scripts/analyse_reconstructions_gaussian_noise.ipynb, lines 11–81 · score 0.60 · phase inverted, Gaussian noise, temporal length constant, stimuli
  4. [4] § Methods › Additional visual stimuli ↔ scripts/reconstruct_videos_gaussian_noise.py, lines 111–162 · score 0.60 · phase inverted, Gaussian noise, temporal length constant, stimuli
  5. [5] § Methods › Mask training ↔ utils_reconstruction/utils_reconstruction.py, lines 15–76 · score 0.57 · alpha blend, background video, layer, masks
  6. [6] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › Not all spatial and temporal frequencies are reconstructed equally ↔ scripts/analyse_reconstructions_drifting_gratings.ipynb, lines 153–289 · score 0.57 · temporal frequencies, temporal length constants, drifting, gratings, square, inverted
  7. [7] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models ↔ scripts/predict_response_from_recon.py, lines 157–239 · score 0.55 · luminance matching, Poisson loss, full video, neurons, activity, correlation
  8. [8] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models ↔ scripts/analyse_reconstructions_from_predicted_activity.ipynb, lines 495–575 · score 0.55 · evaluation mask threshold, Predicted activity, video correlation, diameter, ensembling, gradient
  9. [9] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › Not all spatial and temporal frequencies are reconstructed equally ↔ scripts/analyse_reconstructions_gaussian_noise.ipynb, lines 420–486 · score 0.54 · Gaussian noise, Shannon entropy, motion energy, temporal, frame, reconstructions
  10. [10] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › Not all spatial and temporal frequencies are reconstructed equally ↔ scripts/analyse_reconstructions_drifting_gratings.ipynb, lines 153–289 · score 0.53 · Reconstructed drifting grating, temporal frequencies, interleaved, ground truth, correlation, masked
  11. [11] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › Not all spatial and temporal frequencies are reconstructed equally ↔ scripts/analyse_reconstructions_gaussian_noise.ipynb, lines 211–349 · score 0.53 · Gaussian noise, temporal length constants, square, inverted, ground truth, ensembled
  12. [12] § Results › Video reconstruction using state-of-the-art dynamic neural encoding models › High-quality video reconstruction ↔ scripts/analyse_reconstructions.ipynb, lines 949–972 · score 0.51 · pupil diameter, motion energy, speed, eye, luminance, correlating

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 569 lines · 29 KB · MIT · 4 matches

  1. # %%
  2. import sys
  3. sys.path.append('../')
  4. import numpy as np
  5. import matplotlib.pyplot as plt
  6. import scipy as sp
  8. from utils_reconstruction import image_similarity as imsim
  9. import tifffile
  11. # %%
  12. ## load reconstruction .npy files
  13. num_neurons = [7863, 7908, 8202, 7939, 8122]
  14. mouse_names = [
  15. "dynamic29515-10-12-Video-9b4f6a1a067fe51e15306b9628efea20",
  16. "dynamic29623-4-9-Video-9b4f6a1a067fe51e15306b9628efea20",
  17. "dynamic29647-19-8-Video-9b4f6a1a067fe51e15306b9628efea20",
  18. "dynamic29712-5-9-Video-9b4f6a1a067fe51e15306b9628efea20",
  19. "dynamic29755-2-8-Video-9b4f6a1a067fe51e15306b9628efea20",
  20. ]
  22. max_num_neurons = np.max(num_neurons)
  23. model_list = np.array([0,1,2,3,4,5,6]) # 1,2,3,4,5,6 # 0 held out!
  24. mice = range(0,3)
  25. reps = np.array([0,1,2,3,4])# range(0,5)
  26. spatial_length_constant = range(0,7)
  27. temporal_length_constant = range(0,7)
  28. eval_frame_skip = 30 # default 30
  29. mask_th = 1
  31. video_gt=np.nan*np.ones((len(mice),len(reps)*2,len(spatial_length_constant),len(temporal_length_constant),60,36,64))
  32. video_pred=np.nan*np.ones((model_list.size,len(mice),len(reps)*2,len(spatial_length_constant),len(temporal_length_constant),60,36,64))
  33. responses_pred_gt=np.nan*np.ones((model_list.size,len(mice),len(reps)*2,len(spatial_length_constant),len(temporal_length_constant),max_num_neurons,60))
  34. responses_pred_recon=np.nan*np.ones((model_list.size,len(mice),len(reps)*2,len(spatial_length_constant),len(temporal_length_constant),max_num_neurons,60))
  35. mask=np.nan*np.ones((len(mice),36,64))
  38. for model_n in model_list:
  39. for mouse in mice:
  40. for rep in range(0,len(reps)):#reps:
  41. for spatial_n in spatial_length_constant:
  42. for temporal_n in temporal_length_constant:
  44. datapath=f'../reconstructions/modelfold{model_n}_round4_Gausnoise/reconstruction_summary_m{mouse}_r{reps[rep]}s{spatial_n}t{temporal_n}.npy'
  45. print(datapath)
  46. data = np.load(datapath, allow_pickle=True).item()
  47. if model_n==model_list[0]:
  48. video_gt[mouse,rep,spatial_n,temporal_n] = data['video_gt']
  50. video_pred[model_n-1,mouse,rep,spatial_n,temporal_n] = data['video_pred']
  51. mask[mouse-1] = data['mask'][14:14+36,:]
  52. responses_pred_gt[model_n-1,mouse,rep,spatial_n,temporal_n,0:num_neurons[mouse],:] = data['responses_predicted_gt']
  53. responses_pred_recon[model_n-1,mouse-1,rep,spatial_n,temporal_n,0:num_neurons[mouse],:] = data['responses_predicted_full']
  55. # add phase inverted stimuli as additional reps
  56. datapath=f'../reconstructions/modelfold{model_n}_round5_Gausnoise_PhaseInverted/reconstruction_summary_m{mouse}_r{reps[rep]}s{spatial_n}t{temporal_n}.npy'
  57. print(datapath)
  58. data = np.load(datapath, allow_pickle=True).item()
  59. if model_n==model_list[0]:
  60. video_gt[mouse,rep+len(reps),spatial_n,temporal_n] = data['video_gt']
  62. video_pred[model_n-1,mouse,rep+len(reps),spatial_n,temporal_n] = data['video_pred']
  63. mask[mouse-1] = data['mask'][14:14+36,:]
  64. responses_pred_gt[model_n-1,mouse,rep+len(reps),spatial_n,temporal_n,0:num_neurons[mouse],:] = data['responses_predicted_gt']
  65. responses_pred_recon[model_n-1,mouse-1,rep+len(reps),spatial_n,temporal_n,0:num_neurons[mouse],:] = data['responses_predicted_full']
  67. reps = np.array([0,1,2,3,4,5,6,7,8,9])# range(0,5)
  69. # remove eval grace period
  70. video_gt = video_gt[:,:,:,:,eval_frame_skip:,:,:]
  71. video_pred = video_pred[:,:,:,:,:,eval_frame_skip:,:,:]
  72. responses_pred_gt = responses_pred_gt[:,:,:,:,:,:,eval_frame_skip:]
  73. responses_pred_recon = responses_pred_recon[:,:,:,:,:,:,eval_frame_skip:]
  75. print('video_gt: ', video_gt.shape, video_gt.min(), video_gt.max())
  76. print('video_pred: ', video_pred.shape, video_pred.min(), video_pred.max())
  77. print('mask: ', mask.shape, mask.min(), mask.max())
  79. video_pred = np.clip(video_pred, 0, 255) # not sure i need this... but just in case
  81. video_pred_Amean_all = np.nanmean(video_pred, axis=0) # mean across models, no smoothing
  83. # %%
  84. # correlation between video_gt and video_pred for each rep, spatial and temporal frequency
  85. print('video_gt: ', video_gt.shape, video_gt.min(), video_gt.max())
  86. print('video_pred_Amean_all: ', video_pred_Amean_all.shape, video_pred_Amean_all.min(), video_pred_Amean_all.max())
  88. all_corr = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  89. all_corr_randvid = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant),len(reps)-1))
  91. for mouse in mice:
  92. for rep in range(0,len(reps)):
  93. for spatial_n in spatial_length_constant:
  94. for temporal_n in temporal_length_constant:
  95. all_corr[mouse-1,rep,spatial_n,temporal_n] = imsim.reconstruction_video_corr(video_gt[mouse-1,rep,spatial_n,temporal_n],
  96. video_pred_Amean_all[mouse-1,rep,spatial_n,temporal_n],
  97. np.where(mask[mouse-1] >= mask_th,1,0))
  99. i = -1
  100. for randrep in range(0,len(reps)):
  101. if randrep != rep:
  102. i+=1
  103. all_corr_randvid[mouse-1,rep,spatial_n,temporal_n,i] = imsim.reconstruction_video_corr(video_gt[mouse-1,randrep,spatial_n,temporal_n],
  104. video_pred_Amean_all[mouse-1,rep,spatial_n,temporal_n],
  105. np.where(mask[mouse-1] >= mask_th,1,0))
  106. all_corr_randvid = np.nanmean(all_corr_randvid,axis=-1)
  108. # all corr across mice and reps
  109. all_corr = all_corr.reshape((len(mice)*len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  110. all_corr_randvid = all_corr_randvid.reshape((len(mice)*(len(reps)),len(spatial_length_constant),len(temporal_length_constant)))
  112. # mean corr across mice and reps
  113. mean_corr_ensembled = np.nanmean(all_corr, axis=(0))
  114. mean_corr_randvid_ensembled = np.nanmean(all_corr_randvid, axis=(0))
  115. print('mean_corr: ', mean_corr_ensembled.shape)
  117. all_mean_corr = np.zeros((len(model_list),len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  118. all_mean_corr_randvid = np.zeros((len(model_list),len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant),len(reps)-1))
  120. for model_n in model_list:
  121. for mouse in mice:
  122. for rep in range(0,len(reps)):
  123. for spatial_n in spatial_length_constant:
  124. for temporal_n in temporal_length_constant:
  125. all_mean_corr[model_n-1,mouse-1,rep,spatial_n,temporal_n] = imsim.reconstruction_video_corr(video_gt[mouse-1,rep,spatial_n,temporal_n],
  126. video_pred[model_n-1,mouse-1,rep,spatial_n,temporal_n],
  127. np.where(mask[mouse-1] >= mask_th,1,0))
  128. i = -1
  129. for randrep in range(0,len(reps)):
  130. if randrep != rep:
  131. i+=1
  132. all_mean_corr_randvid[model_n-1,mouse-1,rep,spatial_n,temporal_n,i] = imsim.reconstruction_video_corr(video_gt[mouse-1,randrep,spatial_n,temporal_n],
  133. video_pred[model_n-1,mouse-1,rep,spatial_n,temporal_n],
  134. np.where(mask[mouse-1] >= mask_th,1,0))
  135. all_mean_corr_randvid = np.nanmean(all_mean_corr_randvid,axis=-1)
  137. # all corr across mice and reps
  138. all_mean_corr = np.moveaxis(np.moveaxis(all_mean_corr,(1,2),(0,1)).reshape((len(mice)*len(reps),len(model_list),len(spatial_length_constant),len(temporal_length_constant))),(0),(1))
  139. all_mean_corr_randvid = np.moveaxis(np.moveaxis(all_mean_corr_randvid,(1,2),(0,1)).reshape((len(mice)*len(reps),len(model_list),len(spatial_length_constant),len(temporal_length_constant))),(0),(1))
  140. all_mean_corr = np.nanmean(all_mean_corr, axis=(0))
  141. all_mean_corr_randvid = np.nanmean(all_mean_corr_randvid, axis=(0))
  143. mean_corr = np.nanmean(all_mean_corr, axis=(0))
  145. # %%
  146. ## stats
  148. # ttest between all_corr and all_corr_randvid at every spatial and temporal frequency
  149. pass_corr_essembled = np.zeros((len(spatial_length_constant),len(temporal_length_constant)))
  151. for spatial_n in spatial_length_constant:
  152. for temporal_n in temporal_length_constant:
  153. pass_corr_essembled[spatial_n,temporal_n] = sp.stats.ttest_rel(all_corr[:,spatial_n,temporal_n],all_corr_randvid[:,spatial_n,temporal_n])[1]
  155. print('pass_corr_essembled: ', pass_corr_essembled.shape)
  156. print(pass_corr_essembled<0.001)
  159. # test if difference is significant
  160. # ttest between all_corr and all_corr_randvid at every spatial and temporal frequency
  161. pass_corr_essembled = np.zeros((len(spatial_length_constant),len(temporal_length_constant)))
  163. for spatial_n in spatial_length_constant:
  164. for temporal_n in temporal_length_constant:
  165. pass_corr_essembled[spatial_n,temporal_n] = sp.stats.ttest_rel(all_corr[:,spatial_n,temporal_n],all_mean_corr[:,spatial_n,temporal_n])[1]
  167. print('pass_corr_essemble_improvement: ', pass_corr_essembled.shape)
  168. print(pass_corr_essembled<0.001)
  172. # %%
  173. # contrast adjust images
  174. # for each movie, take the mean image across all time points, then calculate the mean and std of that image
  175. # normalize the predicted images by subtracting the mean and dividing by the std
  176. mouse_n,trial_n,slc,tlc,frame_n, h_n, w_n = video_gt.shape[0],video_gt.shape[1],video_gt.shape[2],video_gt.shape[3],video_gt.shape[4],video_gt.shape[5],video_gt.shape[6]
  178. expanded_mask = np.expand_dims(np.where(mask>= mask_th,1,np.nan) ,axis=(1,2,3,4)).repeat(trial_n,axis=-6).repeat(slc,axis=-5) .repeat(tlc,axis=-4).repeat(frame_n,axis=-3)
  180. # get target mean and std
  182. video_gt_frame_mean = video_gt * expanded_mask
  183. target_means = np.nanmean(video_gt_frame_mean, axis=(-1,-2,-3))
  184. target_stds = np.nanstd(video_gt_frame_mean, axis=(-1,-2,-3))
  186. # get start mean and std
  187. video_pred_frame_mean = video_pred_Amean_all * expanded_mask
  188. start_means = np.nanmean(video_pred_frame_mean, axis=(-1,-2,-3))
  189. start_stds = np.nanstd(video_pred_frame_mean, axis=(-1,-2,-3))
  191. # expand dims of mean and std in time h and w
  192. target_means = np.expand_dims(target_means,axis=(4,5,6)).repeat(frame_n,axis=-3).repeat(h_n,axis=-2).repeat(w_n,axis=-1)
  193. target_stds = np.expand_dims(target_stds,axis=(4,5,6)).repeat(frame_n,axis=-3).repeat(h_n,axis=-2).repeat(w_n,axis=-1)
  194. start_means = np.expand_dims(start_means,axis=(4,5,6)).repeat(frame_n,axis=-3).repeat(h_n,axis=-2).repeat(w_n,axis=-1)
  195. start_stds = np.expand_dims(start_stds,axis=(4,5,6)).repeat(frame_n,axis=-3).repeat(h_n,axis=-2).repeat(w_n,axis=-1)
  197. # # z-score predicted video
  198. video_pred_zscore = (video_pred_Amean_all - start_means) / start_stds
  199. video_pred_norm = (video_pred_zscore*target_stds) + target_means
  200. video_pred_norm = np.clip(video_pred_norm, 0, 255)
  202. # reaply mask
  203. video_gt_masked = video_gt * expanded_mask#np.expand_dims(expanded_mask ,axis=(-3)).repeat(frame_n,axis=-3)
  204. video_pred_Amean_all_masked = video_pred_Amean_all * expanded_mask#p.expand_dims(expanded_mask ,axis=(-3)).repeat(frame_n,axis=-3)
  205. video_pred_norm_masked = video_pred_norm * expanded_mask#np.expand_dims(expanded_mask ,axis=(-3)).repeat(frame_n,axis=-3)
  207. video_gt_masked[np.isnan(video_gt_masked)] = 255/2
  208. video_pred_Amean_all_masked[np.isnan(video_pred_Amean_all_masked)] = 255/2
  209. video_pred_norm_masked[np.isnan(video_pred_norm_masked)] = 255/2
  211. # %%
  212. fig, axs = plt.subplots(3,2, figsize=(15,15))
  214. axs[1,0].imshow(mean_corr_ensembled, cmap='viridis', vmin=0, vmax=1)
  215. axs[1,0].invert_yaxis()
  216. axs[1,0].set_xticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/30).round(2).astype(str))
  217. axs[1,0].set_yticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/3.4).round(2).astype(str))
  218. axs[1,0].set_xlabel('Temporal [sec]')
  219. axs[1,0].set_ylabel('Spatial [deg]')
  220. #set axis square
  221. axs[1,0].set_box_aspect(1)
  222. axs[1,0].set_title('reconstruction correlation')
  223. axs[1,0].tick_params(bottom=True, top=True, left=True, right=True)
  224. for i in range(7):
  225. for j in range(7):
  226. c = mean_corr_ensembled[j,i]
  227. axs[1,0].text(i, j, str(round(c, 2)),
  228. va='center', ha='center')
  230. fig.colorbar(axs[1,1].imshow(mean_corr_ensembled-mean_corr, cmap='viridis', vmin=0, vmax=0.25),ax=axs)
  231. axs[1,1].invert_yaxis()
  232. axs[1,1].set_xticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/30).round(2).astype(str))
  233. axs[1,1].set_yticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/3.4).round(2).astype(str))
  234. axs[1,1].set_xlabel('Temporal [sec]')
  235. axs[1,1].set_ylabel('Spatial [deg]')
  236. #set axis square
  237. axs[1,1].set_box_aspect(1)
  238. axs[1,1].set_title('ensembling improvement')
  239. axs[1,1].tick_params(bottom=True, top=True, left=True, right=True)
  240. for i in range(7):
  241. for j in range(7):
  242. c = mean_corr_ensembled[j,i]-mean_corr[j,i]
  243. axs[1,1].text(i, j, str(round(c, 2)),
  244. va='center', ha='center')
  247. # now relative to pixels and frames
  248. axs[2,0].imshow(mean_corr_ensembled, cmap='viridis', vmin=0, vmax=1)
  249. axs[2,0].invert_yaxis()
  250. axs[2,0].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  251. axs[2,0].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  252. axs[2,0].set_xlabel('Temporal [frames]')
  253. axs[2,0].set_ylabel('Spatial [pix]')
  254. #set axis square
  255. axs[2,0].set_box_aspect(1)
  256. axs[2,0].tick_params(bottom=True, top=True, left=True, right=True)
  257. for i in range(7):
  258. for j in range(7):
  259. c = mean_corr_ensembled[j,i]
  260. axs[2,0].text(i, j, str(round(c, 2)),
  261. va='center', ha='center')
  263. axs[2,1].imshow(mean_corr_ensembled-mean_corr, cmap='viridis', vmin=0, vmax=0.25)
  264. axs[2,1].invert_yaxis()
  265. axs[2,1].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  266. axs[2,1].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  267. axs[2,1].set_xlabel('Temporal [frames]')
  268. axs[2,1].set_ylabel('Spatial [pix]')
  269. #set axis square
  270. axs[2,1].set_box_aspect(1)
  271. axs[2,1].tick_params(bottom=True, top=True, left=True, right=True)
  272. for i in range(7):
  273. for j in range(7):
  274. c = mean_corr_ensembled[j,i]-mean_corr[j,i]
  275. axs[2,1].text(i, j, str(round(c, 2)),
  276. va='center', ha='center')
  280. # apply crop to masks based
  281. mask_expanded = np.expand_dims(mask,axis=(1)).repeat(video_gt.shape[-3],axis=1)
  282. mask_all = np.where(np.sum(np.where(mask >= mask_th,1,0),axis=0) >= 1,1,0)
  283. mask_all_idx = np.where(mask_all == 1)
  284. h_min, h_max = np.min(mask_all_idx[0])-2, np.max(mask_all_idx[0])+2
  285. w_min, w_max = np.min(mask_all_idx[1])-2, np.max(mask_all_idx[1])+2
  286. h = h_max - h_min
  287. w = w_max - w_min
  288. # make square
  289. h_min, h_max = h_min - (w-h)//2, h_max + (w-h)//2
  290. h = h_max - h_min
  291. w = w_max - w_min
  293. print('h_min, h_max, w_min, w_max: ', h_min, h_max, w_min, w_max)
  294. print('h, w: ', h, w)
  296. vid_length = video_gt.shape[-3]
  298. videos_gt_tiled = video_gt[0,0,:,:,:,:,:]
  299. videos_gt_tiled = videos_gt_tiled[:,:,:,h_min:h_max,w_min:w_max]
  300. videos_gt_tiled = videos_gt_tiled[::-1,:,:,:,:] # flip spatial and temporal length constant orders
  301. videos_gt_tiled = np.moveaxis(np.moveaxis(videos_gt_tiled,-2,1).reshape((7*h,7,vid_length,w)),0,-2)
  302. videos_gt_tiled = np.moveaxis(np.moveaxis(videos_gt_tiled,-1,1).reshape((7*w,vid_length,7*h)),0,-1)
  304. videos_gt_tiled_masked = video_gt[0,0,:,:,:,:,:]
  305. videos_gt_tiled_masked = videos_gt_tiled_masked * np.where(mask_expanded[0,-1] >= mask_th,1,0) + (1-np.where(mask_expanded[0,-1] >= mask_th,1,0) )*255/2
  306. videos_gt_tiled_masked= videos_gt_tiled_masked[:,:,:,h_min:h_max,w_min:w_max]
  307. videos_gt_tiled_masked = videos_gt_tiled_masked[::-1,:,:,:,:] # flip spatial and temporal length constant orders
  308. videos_gt_tiled_masked = np.moveaxis(np.moveaxis(videos_gt_tiled_masked,-2,1).reshape((7*h,7,vid_length,w)),0,-2)
  309. videos_gt_tiled_masked = np.moveaxis(np.moveaxis(videos_gt_tiled_masked,-1,1).reshape((7*w,vid_length,7*h)),0,-1)
  311. videos_recon_tiled = video_pred_norm_masked[0,0,:,:,:,:,:]
  312. videos_recon_tiled = videos_recon_tiled * np.where(mask_expanded[0,-1] >= mask_th,1,0) + (1-np.where(mask_expanded[0,-1] >= mask_th,1,0))*255/2
  313. videos_recon_tiled = videos_recon_tiled[::-1,:,:,h_min:h_max,w_min:w_max]
  314. videos_recon_tiled = np.moveaxis(np.moveaxis(videos_recon_tiled,-2,1).reshape((7*h,7,vid_length,w)),0,-2)
  315. videos_recon_tiled = np.moveaxis(np.moveaxis(videos_recon_tiled,-1,1).reshape((7*w,vid_length,7*h)),0,-1)
  316. print('videos_gt_tiled: ', videos_gt_tiled.shape)
  318. axs[0,0].imshow(videos_gt_tiled_masked[-1], cmap='gray')
  319. axs[0,0].set_axis_off()
  320. axs[0,0].set_title('Ground truth')
  322. axs[0,1].imshow(videos_recon_tiled[-1], cmap='gray')
  323. axs[0,1].set_axis_off()
  324. axs[0,1].set_title('Reconstruction')
  326. fig.savefig('../reconstructions/Gaussian_noise_reconstruction.svg', format='svg', dpi=1200)
  328. # tiled video
  329. print('video_gt: ', video_gt.shape)
  330. videos_gt_tiled_masked = video_gt[0,:,:,:,:,:,:]
  331. videos_gt_tiled_masked = videos_gt_tiled_masked * np.where(mask_expanded[0,-1] >= mask_th,1,0) + (1-np.where(mask_expanded[0,-1] >= mask_th,1,0) )*255/2
  332. videos_gt_tiled_masked= videos_gt_tiled_masked[:,::-1,:,:,h_min:h_max,w_min:w_max]
  333. videos_recon_tiled = video_pred_norm_masked[0,:,:,:,:,:,:]
  334. videos_recon_tiled = videos_recon_tiled * np.where(mask_expanded[0,-1] >= mask_th,1,0) + (1-np.where(mask_expanded[0,-1] >= mask_th,1,0))*255/2
  335. videos_recon_tiled = videos_recon_tiled[:,::-1,:,:,h_min:h_max,w_min:w_max]
  339. videos_interleaved = np.concatenate((videos_gt_tiled_masked[:,None,:],videos_recon_tiled[:,None,:]), axis=1)
  340. videos_interleaved = np.moveaxis(np.moveaxis(videos_interleaved,-3,1).reshape((len(reps)*vid_length,2,7,7,h,w)),0,-3) # concatenate directions
  341. videos_interleaved = np.moveaxis(np.moveaxis(videos_interleaved,0,1).reshape((2*7,7,len(reps)*vid_length,h,w)),0,0) # tile gt and recon
  342. videos_interleaved = np.moveaxis(np.moveaxis(videos_interleaved,-2,1).reshape((2*7*h,7,len(reps)*vid_length,w)),0,-2) # tile sf
  343. videos_interleaved = np.moveaxis(np.moveaxis(videos_interleaved,-1,1).reshape((7*w,len(reps)*vid_length,7*h*2)),0,-1) # tile tf
  345. # save video as tiff
  346. tifffile.imwrite('../reconstructions/Gaussian_noise.tiff',
  347. videos_interleaved.astype('uint8'),
  348. imagej=True,
  349. metadata = {'unit': 'um','fps': 30.0,'axes': 'TYX',})
  353. # %%
  354. videos_gt_tiled_hyperstack = np.concatenate((videos_gt_tiled_masked[:,:,:,None],videos_recon_tiled[:,:,:,None]), axis=3).reshape(-1,2,30,30,31)
  355. videos_gt_tiled_hyperstack = np.moveaxis(videos_gt_tiled_hyperstack,(0,1,2,3,4),(1,2,0,3,4))
  356. print(videos_gt_tiled_hyperstack.shape)
  358. # # # save video as tiff
  359. tifffile.imwrite('../reconstructions/Gaussian_noise_hyperstack.tiff',
  360. videos_gt_tiled_hyperstack.astype('uint16'),
  361. imagej=True,
  362. metadata = {'unit': 'um','fps': 30.0,'axes': 'TZCYX',})
  364. # %% [markdown]
  365. # # Reconstructed entropy
  367. # %%
  368. # check this approach to calc entropy makes sense
  370. def video_entropy_in_space(video, mask):
  371. idx_inmask = mask.flatten() == 1
  373. img_entropy_all=[]
  374. for frame in range(video.shape[0]):
  375. video_frame = video[frame].flatten()
  376. video_frame = video_frame[idx_inmask]
  377. hist, _ = np.histogram(video_frame, bins=256//10, range=(0, 256), density=True)
  378. hist = hist[hist > 0]
  379. img_entropy = sp.stats.entropy(hist, base=2)
  380. img_entropy_all.append(img_entropy)
  381. img_entropy_all = np.array(img_entropy_all)
  382. return np.nanmean(img_entropy_all)#,corr_all
  384. def video_energy(video, mask):
  385. # ground_truth: (time, height, width)
  386. # mask: (height, width) or (time, height, width)
  387. # returns: (time, height, width)
  389. if len(mask.shape) == 3:
  390. mask = mask[0, :, :] # make 2D
  391. idx_inmask = mask.flatten() == 1
  392. video.swapaxes(0, -1) # height, width,time
  393. video = video.reshape(video.shape[0],-1) # flatten spatial dimensions
  394. video.swapaxes(-1, 0) # time, height*width
  395. video = video[:,idx_inmask] # mask spatial dimensions
  396. frame_energy = np.mean(np.abs(np.diff(video, axis=0)),axis=(1)) # sum of absolute differences (motion energy)
  398. return frame_energy
  400. # %%
  401. Entropy_in_space_gt = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  402. # Entropy_in_timenspace_gt = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  403. Motion_energy_gt = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  404. Entropy_in_space_recon = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  405. # Entropy_in_timenspace_recon = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  406. Motion_energy_recon = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  408. for mouse in mice:
  409. for rep in range(0,len(reps)):
  410. for spatial_n in spatial_length_constant:
  411. for temporal_n in temporal_length_constant:
  412. Entropy_in_space_gt[mouse,rep,spatial_n,temporal_n] = video_entropy_in_space(video_gt[mouse,rep,spatial_n,temporal_n,:,:,11:11+36],np.where(mask[mouse,:,11:11+36] >= 0,1,0))
  413. # Entropy_in_timenspace_gt[mouse,rep,spatial_n,temporal_n] = video_entropy_in_timenspace(video_gt[mouse,rep,spatial_n,temporal_n,:,:,11:11+36],np.where(mask[mouse,:,11:11+36] >= 0,1,0))
  414. Motion_energy_gt[mouse,rep,spatial_n,temporal_n] = video_energy(video_gt[mouse,rep,spatial_n,temporal_n,:,:,11:11+36],np.where(mask[mouse,:,11:11+36] >= 0,1,0)).mean()
  416. Entropy_in_space_recon[mouse,rep,spatial_n,temporal_n] = video_entropy_in_space(video_pred_norm_masked[mouse,rep,spatial_n,temporal_n],np.where(mask[mouse] >= mask_th,1,0))
  417. # Entropy_in_timenspace_recon[mouse,rep,spatial_n,temporal_n] = video_entropy_in_timenspace(video_pred_norm_masked[mouse,rep,spatial_n,temporal_n],np.where(mask[mouse] >= mask_th,1,0))
  418. Motion_energy_recon[mouse,rep,spatial_n,temporal_n] = video_energy(video_pred_norm_masked[mouse,rep,spatial_n,temporal_n],np.where(mask[mouse] >= mask_th,1,0)).mean()
  420. # %%
  421. fig, ax = plt.subplots(2,2, figsize=(10,10))
  422. ax[0,0].imshow(Entropy_in_space_gt.mean(axis=(0,1)), cmap='viridis', vmin=2.5, vmax=4.5)
  423. ax[0,0].invert_yaxis()
  424. ax[0,0].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  425. ax[0,0].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  426. ax[0,0].set_xlabel('Temporal [frames]')
  427. ax[0,0].set_ylabel('Spatial [pix]')
  428. #set axis square
  429. ax[0,0].set_box_aspect(1)
  430. ax[0,0].tick_params(bottom=True, top=True, left=True, right=True)
  431. for i in range(7):
  432. for j in range(7):
  433. c = Entropy_in_space_gt.mean(axis=(0,1))[j,i]
  434. ax[0,0].text(i, j, str(round(c, 2)),
  435. va='center', ha='center')
  436. ax[0,0].set_title('Shannon Entropy in space GT')
  438. ax[0,1].imshow(Motion_energy_gt.mean(axis=(0,1)), cmap='viridis', vmin=0, vmax=75)
  439. ax[0,1].invert_yaxis()
  440. ax[0,1].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  441. ax[0,1].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  442. ax[0,1].set_xlabel('Temporal [frames]')
  443. ax[0,1].set_ylabel('Spatial [pix]')
  444. #set axis square
  445. ax[0,1].set_box_aspect(1)
  446. ax[0,1].tick_params(bottom=True, top=True, left=True, right=True)
  447. for i in range(7):
  448. for j in range(7):
  449. c = Motion_energy_gt.mean(axis=(0,1))[j,i]
  450. ax[0,1].text(i, j, str(round(c, 2)),
  451. va='center', ha='center')
  452. ax[0,1].set_title('Motion energy GT')
  454. ax[1,0].imshow(Entropy_in_space_recon.mean(axis=(0,1)), cmap='viridis', vmin=2.5, vmax=4.5)
  455. ax[1,0].invert_yaxis()
  456. ax[1,0].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  457. ax[1,0].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  458. ax[1,0].set_xlabel('Temporal [frames]')
  459. ax[1,0].set_ylabel('Spatial [pix]')
  460. #se axis square
  461. ax[1,0].set_box_aspect(1)
  462. ax[1,0].tick_params(bottom=True, top=True, left=True, right=True)
  463. for i in range(7):
  464. for j in range(7):
  465. c = Entropy_in_space_recon.mean(axis=(0,1))[j,i]
  466. ax[1,0].text(i, j, str(round(c, 2)),
  467. va='center', ha='center')
  468. ax[1,0].set_title('Shannon Entropy in space Recon')
  470. ax[1,1].imshow(Motion_energy_recon.mean(axis=(0,1)), vmin=0, vmax=75)
  471. ax[1,1].invert_yaxis()
  472. ax[1,1].set_xticks(np.arange(7), ['0','1','2','4','8','16','32'])
  473. ax[1,1].set_yticks(np.arange(7), ['0','1','2','4','8','16','32'])
  474. ax[1,1].set_xlabel('Temporal [frames]')
  475. ax[1,1].set_ylabel('Spatial [pix]')
  476. #set axis square
  477. ax[1,1].set_box_aspect(1)
  478. ax[1,1].tick_params(bottom=True, top=True, left=True, right=True)
  479. for i in range(7):
  480. for j in range(7):
  481. c = Motion_energy_recon.mean(axis=(0,1))[j,i]
  482. ax[1,1].text(i, j, str(round(c, 2)),
  483. va='center', ha='center')
  484. ax[1,1].set_title('Motion energy recon')
  486. fig.savefig('../reconstructions/Gaussian_noise_entropy.svg', format='svg', dpi=1200)
  489. # %% [markdown]
  490. # # Phase inverted stimuli
  492. # %%
  493. all_corr_same_phase = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  494. all_corr_opposite_phase = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  495. all_corr_opposite_phase_recontorecon = np.zeros((len(mice),len(reps),len(spatial_length_constant),len(temporal_length_constant)))
  497. for mouse in mice:
  498. for rep in range(0,len(reps)):
  499. if rep > len(reps)//2-1: # phase 2
  500. rep_opposite_phase = rep - len(reps)//2
  501. elif rep <= len(reps)//2-1: # phase 1
  502. rep_opposite_phase = rep + len(reps)//2
  504. for spatial_n in spatial_length_constant:
  505. for temporal_n in temporal_length_constant:
  506. all_corr_same_phase[mouse-1,rep,spatial_n,temporal_n] = imsim.reconstruction_video_corr(video_gt[mouse-1,rep,spatial_n,temporal_n],
  507. video_pred_Amean_all[mouse-1,rep,spatial_n,temporal_n],
  508. np.where(mask[mouse-1] >= mask_th,1,0))
  509. all_corr_opposite_phase[mouse-1,rep,spatial_n,temporal_n] = imsim.reconstruction_video_corr(video_gt[mouse-1,rep_opposite_phase,spatial_n,temporal_n],
  510. video_pred_Amean_all[mouse-1,rep,spatial_n,temporal_n],
  511. np.where(mask[mouse-1] >= mask_th,1,0))
  513. all_corr_opposite_phase_recontorecon[mouse-1,rep,spatial_n,temporal_n] = imsim.reconstruction_video_corr(video_pred_Amean_all[mouse-1,rep_opposite_phase,spatial_n,temporal_n],
  514. video_pred_Amean_all[mouse-1,rep,spatial_n,temporal_n],
  515. np.where(mask[mouse-1] >= mask_th,1,0))
  517. # %%
  518. fig, ax = plt.subplots(2,2, figsize=(10,10))
  519. ax[0,0].imshow(all_corr_same_phase.mean(axis=(0,1)), cmap='RdYlGn', vmin=-1, vmax=1)
  520. ax[0,0].invert_yaxis()
  521. ax[0,0].set_xticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/30).round(2).astype(str))
  522. ax[0,0].set_yticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/3.4).round(2).astype(str))
  523. ax[0,0].set_xlabel('Temporal [sec]')
  524. ax[0,0].set_ylabel('Spatial [deg]')
  525. ax[0,0].set_title('corr between recon to same phase GT')
  526. ax[0,0].tick_params(bottom=True, top=True, left=True, right=True)
  527. for i in range(7):
  528. for j in range(7):
  529. c = all_corr_same_phase.mean(axis=(0,1))[j,i]
  530. ax[0,0].text(i, j, str(round(c, 2)),
  531. va='center', ha='center')
  534. ax[0,1].imshow(all_corr_opposite_phase.mean(axis=(0,1)), cmap='RdYlGn', vmin=-1, vmax=1)
  535. ax[0,1].invert_yaxis()
  536. ax[0,1].set_xticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/30).round(2).astype(str))
  537. ax[0,1].set_yticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/3.4).round(2).astype(str))
  538. ax[0,1].set_xlabel('Temporal [sec]')
  539. ax[0,1].set_ylabel('Spatial [deg]')
  540. ax[0,1].set_title('corr between recon to opposite phase GT')
  541. ax[0,1].tick_params(bottom=True, top=True, left=True, right=True)
  542. for i in range(7):
  543. for j in range(7):
  544. c = all_corr_opposite_phase.mean(axis=(0,1))[j,i]
  545. ax[0,1].text(i, j, str(round(c, 2)),
  546. va='center', ha='center')
  549. ax[1,0].imshow(all_corr_opposite_phase_recontorecon.mean(axis=(0,1)), cmap='RdYlGn', vmin=-1, vmax=1)
  550. ax[1,0].invert_yaxis()
  551. ax[1,0].set_xticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/30).round(2).astype(str))
  552. ax[1,0].set_yticks(np.arange(7), np.array([0,1,2,4,8,16,32]).__mul__(1/3.4).round(2).astype(str))
  553. ax[1,0].set_xlabel('Temporal [sec]')
  554. ax[1,0].set_ylabel('Spatial [deg]')
  555. ax[1,0].set_title('corr between recon to recon of opposite phase GT')
  556. ax[1,0].tick_params(bottom=True, top=True, left=True, right=True)
  557. for i in range(7):
  558. for j in range(7):
  559. c = all_corr_opposite_phase_recontorecon.mean(axis=(0,1))[j,i]
  560. ax[1,0].text(i, j, str(round(c, 2)),
  561. va='center', ha='center')
  565. cbar1 = plt.colorbar(ax[1,0].images[0], ax=ax[1,0], orientation='vertical')
  567. fig.savefig('../reconstructions/Gaussian_noise_PhaseInversion.svg', format='svg', dpi=1200)

analyse_reconstructions_gaussian_noise.ipynb at commit eb5ad21, under MIT · at the source

Overview

Authors: Joel Bauer1,2, Troy W Margrie1, Claudia Clopath1,2
ORCID iDs: Joel Bauer, Troy W Margrie, Claudia Clopath
  1. Sainsbury Wellcome Centre, University College London London United Kingdom
  2. Bioengineering Dept., Imperial College London United Kingdom
Institutions: Sainsbury Wellcome Centre (United Kingdom); Imperial College London (United Kingdom)
Journal: eLife, volume 14, article RP105081
Dates: published online 10 March 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.7554/elife.105081 · PMID 41804097 · PMCID PMC12975128 · OpenAlex W4408807840
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: optical imaging (calcium, voltage, 2-photon) (modality), mouse (organism), systems (subfield)
Methods: Connectivity, Machine learning, Physiology & signal measures
Keywords: vision, visual cortex, sensory reconstruction, deep learning, perception, calcium imaging, Mouse
MeSH: Image Processing, Computer-Assisted*, Neurons*, Visual Cortex*, Animals, Mice, Photic Stimulation (* major topic)
Journal subjects: Neuroscience
Topic: Neural dynamics and brain function (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: Wellcome Trust (318818/Z/24/Z, 10.35802/306384, 10.35802/200790); Gatsby Charitable Foundation (GAT4057); European Molecular Biology Organization (ALTF 415-2024); Simons Foundation (564408); Engineering and Physical Sciences Research Council (EP/R035806/1); European Research Council (MotorAdapt 101169605)
Citations: cited by 2 papers (Europe PMC); 66 references in the paper

Abstract

The ability to reconstruct images represented by the brain has the potential to give us an intuitive understanding of what the brain sees. Reconstruction of visual input from human fMRI data has garnered significant attention in recent years. Comparatively less focus has been directed towards vision reconstruction from single-cell recordings, despite its potential to provide a more direct measure of the information represented by the brain. Here, we achieve high-quality reconstructions of natural movies presented to mice, from the activity of neurons in their visual cortex for the first time. Using our method of video optimization via backpropagation through a state-of-the-art dynamic neural encoding model, we reliably reconstruct 10 s movies at 30 Hz from two-photon calcium imaging data. We achieve a pixel-level correlation of 0.57 between ground-truth movies and single-trial reconstructions. Previous reconstructions based on awake mouse V1 neuronal responses to static images achieved a pixel-level correlation of 0.24 over a similar retinotopic area. We find that critical for high-quality reconstructions are the number of neurons in the dataset and the use of model ensembling. This paves the way for movie reconstruction to be used as a tool to investigate a variety of visual processing phenomena.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 12 matches between paragraphs and lines of code.

lRomul/sensorium

License: MIT
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: 6849050e74e2843fbd70b33af110d71638aa37bb, 22 November 2023
Languages: Python (24)
Size: 39 files, 24 scripts
Software Heritage: not archived
Found in: the text, “State-of-the-art dynamic neural encoding model”
Holds: README, license file, CITATION.cff, environment (Dockerfile, requirements.txt, setup.cfg)
Not found: tests, continuous integration, documentation
Tools: NumPy (13 files), PyTorch (12 files), pandas (1 file), Pillow (1 file)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
26 files
At the source: github.com/lRomul/sensorium

Joel-Bauer/movie_reconstruction_code

License: MIT
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: eb5ad2143b5d54aad4ca44cb26b919cfab987920, 9 January 2026
Languages: Python (11), Jupyter (5)
Size: 30 files, 16 scripts
Software Heritage: not archived
Found in: “Data availability”
Holds: README, license file, 5 notebooks
Not found: CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (13 files), Matplotlib (12 files), PyTorch (8 files), tifffile (6 files), SciPy (5 files), OpenCV (2 files)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
18 files
At the source: github.com/Joel-Bauer/movie_reconstruction_code

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 40 scripts, each with its path and the digest of its content;
  • 12 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data availability

The code is available at https://github.com/Joel-Bauer/movie_reconstruction_code (copy archived at Bauer, 2025).

The following previously published datasets were used:

FaheyP TurishchevaP HanselL FroebeR PonderK VystrcilováM QiuY WillekeK BashiriM ToliasA SinzA EckerA 2023The Dynamic Sensorium competition for predicting large-scale mouse visual cortex activity from videos - DatasetG-Node GinSensorium2023Data

FaheyP TurishchevaP HanselL FroebeR PonderK VystrcilováM QiuY WillekeK BashiriM ToliasA SinzA EckerA 2023The Dynamic Sensorium competition for predicting large-scale mouse visual cortex activity from videos - DatasetG-Node Ginsensorium_2023_data

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 30 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 3 authors, 7 keywords, 6 MeSH terms, 6 funders, 62 references.

Cite

This paper

Bauer, J., Margrie, T. W., & Clopath, C. (2026). Movie reconstruction from mouse visual cortex activity. eLife, 14, RP105081. https://doi.org/10.7554/elife.105081

BibTeX

@article{bauer2026movie,
author = {Bauer, Joel and Margrie, Troy W and Clopath, Claudia},
title = {{Movie reconstruction from mouse visual cortex activity}},
journal = {eLife},
year = {2026},
month = mar,
volume = {14},
pages = {RP105081},
publisher = {eLife Sciences Publications, Ltd},
issn = {2050-084X},
doi = {10.7554/elife.105081},
url = {https://doi.org/10.7554/elife.105081},
pmid = {41804097},
pmcid = {PMC12975128}
}

RIS

TY - JOUR
AU - Bauer, Joel
AU - Margrie, Troy W
AU - Clopath, Claudia
TI - Movie reconstruction from mouse visual cortex activity
T2 - eLife
J2 - Elife
PY - 2026
DA - 2026/03/10
VL - 14
SP - RP105081
SN - 2050-084X
PB - eLife Sciences Publications, Ltd
DO - 10.7554/elife.105081
UR - https://doi.org/10.7554/elife.105081
LA - en
ER -

CSL-JSON

{
"id": "10.7554/elife.105081",
"type": "article-journal",
"title": "Movie reconstruction from mouse visual cortex activity",
"container-title": "eLife",
"author": [
{
"family": "Bauer",
"given": "Joel"
},
{
"family": "Margrie",
"given": "Troy W"
},
{
"family": "Clopath",
"given": "Claudia"
}
],
"container-title-short": "Elife",
"volume": "14",
"page": "RP105081",
"DOI": "10.7554/elife.105081",
"PMID": "41804097",
"PMCID": "PMC12975128",
"ISSN": "2050-084X",
"publisher": "eLife Sciences Publications, Ltd",
"URL": "https://doi.org/10.7554/elife.105081",
"language": "en",
"issued": {
"date-parts": [
[
2026,
3,
10
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1002/advs.202520220 [code]
Learnable Diffusion Framework for Mouse V1 Neural Decoding.
Journal: Advanced science (Weinheim, Baden-Wurttemberg, Germany)
In common: OpenCV, Pillow, PyTorch, 4 other tools, systems, mouse, 6 references
[2] doi:10.1371/journal.pcbi.1014263 [code]
MIRAGE: Robust multi-modal architectures translate fMRI-to-image models from vision to mental imagery.
Journal: PLoS computational biology
In common: OpenCV, Pillow, PyTorch, 4 other tools, 5 references
[3] doi:10.7554/elife.107933 [code]
Modality-agnostic decoding of vision and language from fMRI.
Journal: eLife
In common: OpenCV, Pillow, PyTorch, 4 other tools, 5 references
[4] doi:10.1016/j.isci.2026.116206 [code]
Gut distension evokes rapid neural dynamics in vagal and hindbrain populations of larval zebrafish.
Journal: iScience
In common: tifffile, OpenCV, Pillow, 5 other tools, optical imaging (calcium, voltage, 2-photon), systems, 2 references
[5] doi:10.1016/j.patter.2026.101538 [code]
A multi-modal foundation model for brain disease diagnosis and medical imaging.
Journal: Patterns (New York, N.Y.)
In common: tifffile, OpenCV, Pillow, 5 other tools, 2 references
[6] doi:10.1007/s12021-026-09803-3 [code]
NeuroFusion: A Unified Framework for Generalized Visual Stimulus Decoding from fMRI Across Datasets and Subjects.
Journal: Neuroinformatics
In common: Pillow, PyTorch, pandas, 3 other tools, 4 references
[7] doi:10.1523/eneuro.0459-25.2026 [code]
Open-Source Platform for Adjustable Training Regimes in Freely Moving and Head-Fixed Mice.
Journal: eNeuro
In common: OpenCV, Pillow, PyTorch, 2 other tools, optical imaging (calcium, voltage, 2-photon), mouse, 3 references
[8] doi:10.1016/j.crmeth.2026.101476 [code]
Unsupervised deep learning enables blur-free resolution enhancement in two-photon microscopy.
Journal: Cell reports methods
In common: tifffile, OpenCV, PyTorch, 4 other tools, optical imaging (calcium, voltage, 2-photon), mouse, 2 references
[9] doi:10.1364/boe.600665 [code]
NeuroSeg-MF: robust neuron segmentation in two-photon Ca&lt;sup&gt;2+&lt;/sup&gt; imaging using multi-feature fusion and detection-guided SAM.
Journal: Biomedical optics express
In common: tifffile, OpenCV, Pillow, 5 other tools, optical imaging (calcium, voltage, 2-photon), 1 reference
[10] doi:10.1111/ejn.70582 [code]
Multifiber Array-Based Photometry System for Multiregional Functional Mapping in the Mouse Brain.
Journal: The European journal of neuroscience
In common: tifffile, OpenCV, Pillow, 4 other tools, optical imaging (calcium, voltage, 2-photon), systems, mouse, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.