OSCR

Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks.

Code ↔ Paper

4 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 4 matches · 1 of them tie a paragraph to a whole file, not to given lines: a weak match, whose lines are not tinted
  1. [1] § Methods › Analysis › GLM component of the encoding model analysis ↔ Code/GLM/S05_design_matrix_layer_x.m, the whole file · a weak match · score 0.77 · design matrix, SPM, downsampled, HRF, filter, ReLU
  2. [2] § Methods › Analysis › GLM component of the encoding model analysis ↔ Code/GLM/S07_layer_x_cross_validation.m, lines 1–60 · score 0.55 · cross validated, SPM, GLMs, correlation, voxel, layer
  3. [3] § Methods › Experimental design and task › Task implementation and data collection ↔ Code/Task/gameplay/Gameplay_Enduro.py, lines 94–153 · score 0.53 · pixel screen, Arcade Learning Environment, height, width, frames, rewards
  4. [4] § Methods › Analysis › DQN-based encoding model analysis ↔ Code/GLM/S07_layer_x_cross_validation.m, lines 1–60 · score 0.50 · cross validation, GLM, correlation, voxel, layers, model

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

MATLAB · 154 lines · 4.9 KB · MIT · 2 matches

  1. function S07_layer_x_cross_validation(subject, game, DQN, reg_parameter)
  3. root_path = 'YOUR_DATA_PATH';
  4. main_dir = strcat(subject, '/', game) ;
  6. sess_no = 5;
  8. GLM_name = 'GLMs_Layer_x';
  10. lambda_seq = reg_parameter;
  12. load(strcat(root_path, main_dir, '/GLMs/GLM_empty/SPM.mat'));
  14. beta_template_nii = load_nii(strcat(root_path, main_dir, '/GLMs/GLM_empty/beta_0001.nii'));
  16. NNZ_per_model_all = zeros(1,sess_no);
  18. load(strcat(root_path, main_dir, '/GLMs/', GLM_name, '/', DQN, '/GLM_sess_1/lambda_all_beta_all_sparse.mat'), 'xyz_vec');
  20. N_voxels_in_mask = size(xyz_vec,1);
  22. beta_value_all_runs = NaN(max(NNZ_per_model_all), N_voxels_in_mask, size(lambda_seq,2), sess_no, 'single');
  23. beta_ind_all_runs = zeros(max(NNZ_per_model_all), N_voxels_in_mask, size(lambda_seq,2), sess_no, 'uint16');
  24. intercept_all_runs = zeros(N_voxels_in_mask, size(lambda_seq,2), sess_no);
  26. for run_no = 1:sess_no
  28. clear 'NNZ_per_model'
  29. clear xyz_vec
  30. clear beta_ind_mat
  31. clear beta_value_mat
  33. load(strcat(root_path, main_dir, '/GLMs/', GLM_name, '/', DQN, '/GLM_sess_', num2str(run_no), '/lambda_all_beta_all_sparse.mat'),...
  34. 'NNZ_per_model', 'xyz_vec', 'beta_ind_mat', 'beta_value_mat', 'intercept');
  36. beta_value_all_runs(1:NNZ_per_model_all(1,run_no),:,:,run_no) = beta_value_mat;
  37. beta_ind_all_runs(1:NNZ_per_model_all(1,run_no),:,:,run_no) = beta_ind_mat;
  38. intercept_all_runs(:,:,run_no) = intercept;
  40. end
  42. clear beta_ind_mat
  43. clear beta_value_mat
  44. clear intercept
  46. corr_r_all_runs = NaN(N_voxels_in_mask, size(lambda_seq,2), sess_no);
  48. for run_no = 1:sess_no
  50. disp(run_no);
  52. res_nii = load_nii(strcat(root_path, main_dir, '/GLMs/GLM_empty/Res_all.nii'));
  53. res_ind = SPM.xX.K(run_no).row(2:end-1)';
  54. res_nii_img = NaN(size(res_ind, 1), N_voxels_in_mask);
  56. for vx_count = 1:N_voxels_in_mask
  58. x = xyz_vec(vx_count,1);
  59. y = xyz_vec(vx_count,2);
  60. z = xyz_vec(vx_count,3);
  62. res_nii_img(:,vx_count) = squeeze(res_nii.img(x,y,z,res_ind));
  64. end
  66. clear res_nii
  67. clear X_hpf
  69. load(strcat(root_path, main_dir, '/GLMs/', GLM_name, '/', DQN, '/GLM_sess_', num2str(run_no), '/X_hpf.mat'));
  70. X_hpf = zscore(X_hpf(:,1:end-1),1);
  72. train_bin = true(sess_no,1);
  73. train_bin(run_no,1) = false;
  75. beta_value_train = beta_value_all_runs(:,:,:,train_bin);
  76. beta_ind_train = beta_ind_all_runs(:,:,:,train_bin);
  77. intercept_train = intercept_all_runs(:,:,train_bin);
  79. corr_r_curr_run_mat = NaN(N_voxels_in_mask, size(lambda_seq,2));
  81. for vx_count = 1:N_voxels_in_mask
  83. res_ts = res_nii_img(:, vx_count);
  85. res_ts = zscore(res_ts,1);
  87. beta_full_mat_all_lambda = NaN(size(X_hpf,2), size(lambda_seq,2));
  89. for lambda_no = 1:size(lambda_seq,2)
  91. beta_full_lambda_curr = zeros(size(X_hpf,2), sess_no-1, 'single');
  93. for train_run_no = 1:sess_no-1
  95. beta_value_vec_lambda_curr_train_run = squeeze(beta_value_train(:,vx_count,lambda_no,train_run_no));
  96. beta_ind_vec_lambda_curr_train_run = squeeze(beta_ind_train(:,vx_count,lambda_no,train_run_no));
  98. beta_value_vec_lambda_curr_train_run(beta_ind_vec_lambda_curr_train_run == 0,:) = [];
  99. beta_ind_vec_lambda_curr_train_run(beta_ind_vec_lambda_curr_train_run == 0,:) = [];
  101. beta_full_lambda_curr(beta_ind_vec_lambda_curr_train_run, train_run_no) = beta_value_vec_lambda_curr_train_run;
  102. end
  104. beta_full_lambda_curr = double(beta_full_lambda_curr);
  106. beta_full_lambda_curr_mean = mean(beta_full_lambda_curr,2);
  108. beta_full_mat_all_lambda(:,lambda_no) = beta_full_lambda_curr_mean;
  110. end
  112. intercept_all_runs_mean = zeros(1, size(lambda_seq,2));
  114. intercept_all_runs_mean(1,:) = mean(intercept_train(vx_count,:,:),3);
  116. y_pred_all_lambda = X_hpf*beta_full_mat_all_lambda + ones(size(X_hpf,1),1)*intercept_all_runs_mean;
  118. corr_r_all_lambda = corrcoef([res_ts, y_pred_all_lambda]);
  120. corr_r_curr_run_mat(vx_count,:) = corr_r_all_lambda(1, 2:end);
  122. end
  124. corr_r_all_runs(:,:,run_no) = corr_r_curr_run_mat;
  127. end
  129. corr_r_all_runs_mean = mean(corr_r_all_runs,3);
  131. for lambda_no = 1:size(lambda_seq,2)
  133. corr_3D = NaN(79,95,79);
  135. for vx_count = 1:N_voxels_in_mask
  137. x = xyz_vec(vx_count,1);
  138. y = xyz_vec(vx_count,2);
  139. z = xyz_vec(vx_count,3);
  141. corr_3D(x,y,z) = corr_r_all_runs_mean(vx_count, lambda_no);
  143. end
  145. beta_template_nii.img = single(corr_3D);
  147. save_nii(beta_template_nii, strcat(root_path, main_dir, '/GLMs/', GLM_name, '/', DQN, '/lambda_', num2str(lambda_no, '%04.0f'), '_correlation_map_layer_x'));
  149. end
  151. disp(' ');
  152. disp('Done!');
  153. disp(' ');

S07_layer_x_cross_validation.m at commit 8c64aab, under MIT · at the source

Overview

Authors: Sabine Haberland1, Hannes Ruge1, Holger Frimmel1
ORCID iDs: Sabine Haberland, Hannes Ruge
  1. Institut of General Psychology, TUD Dresden University of Technology, Dresden, Germany
Institutions: Technische Universität Dresden (Germany)
Journal: Imaging neuroscience (Cambridge, Mass.), volume 4, article IMAG.a.1142
Dates: received 19 September 2025; accepted 24 January 2026; published online 16 March 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1162/imag.a.1142 · PMID 41852943 · PMCID PMC12994000 · OpenAlex W7126231831
Open access: diamond, a free copy (OpenAlex)
Status: code verified
Categories: fMRI (modality), human (organism), systems (subfield)
Methods: Connectivity, Statistics, Machine learning, Preprocessing, fMRI & imaging
Keywords: deep neural networks, encoding models, arcade games, neuroimaging, LSTM and fMRI
Topic: Action Observation and Synchronization (Social Psychology, Psychology), according to OpenAlex
Funding: Deutsche Forschungsgemeinschaft (DFG) (445383113)
Citations: cited by 1 paper (Europe PMC); 89 references in the paper

Abstract

Human behavior arises from the continuous transformation of sensory input into goal-directed actions. While existing analytical methods often break time into discrete events, the stages and underlying representations involved in stimulus-response (S-R) transformations within time-continuous, complex environments remain incompletely understood. Encoding models, combined with deep neural networks (DNNs) for feature generation, offer a promising framework for capturing these neural processes. While DNNs continue to improve in performance, it remains unclear whether these advances translate into closer alignment with human cognitive mechanisms. To address this, we collected fMRI data from participants (N = 23) as they played arcad video games and used DNN-based encoding models to predict human brain activity. We compared the prediction accuracy of features from three DNNs at different stages of development within our encoding model. The results show that the most advanced DNN provides the most predictive feature space for neural responses, while also revealing a closer hierarchical alignment between its internal representations and the brain’s functional organization. These results enable a more fine-grained characterization of time-continuous S-R transformations in high-dimensional visuomotor tasks, progressing along the dorsal visual stream and extending into motor-related regions. This approach highlights the potential of machine learning to advance cognitive neuroscience by enhancing the investigation of ecological valid experimental tasks.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.

SHaberland15/Arcade_DQN_Research_fMRI

License: MIT
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: 8c64aab8fc34ec40d5304e189a922ce94d801748, 17 December 2025
Languages: Python (12), MATLAB (6)
Size: 55 files, 18 scripts
Software Heritage: not archived
Found in: “Data and Code Availability”
Holds: README, license file, environment (Code/Task/dockerfile, Code/DQN/Ape-X/dockerfile, Code/DQN/Baseline_DQN/dockerfile, Code/DQN/SEED/dockerfile)
Not found: CITATION.cff, tests, continuous integration, documentation
Tools: NumPy (10 files), Pillow (8 files), PyTorch (8 files), scikit-image (8 files), Statistics and Machine Learning Toolbox (3 files), SPM (3 files), Tools for NIfTI and ANALYZE image (MATLAB) (2 files), OpenCV (2 files), TensorFlow (2 files)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
20 files
At the source: github.com/SHaberland15/Arcade_DQN_Research_fMRI

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 18 scripts, each with its path and the digest of its content;
  • 4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data and Code Availability

All behavioral data, the recorded screen observed by the subjects, as well as the results of our analysis on single-subject and group-analysis level are publicly available at https://osf.io/9cwq4/ (DOI: 10.17605/OSF.IO/9CWQ4 (https://doi.org/10.17605/OSF.IO/9CWQ4)). The experimental tasks, the DQN code used to generate the features, and all analysis scripts are accessible via the public GitHub repository at https://github.com/SHaberland15/Arcade_DQN_Research_fMRI. A permanent DOI for the repository was created via Zenodo: 10.5281/zenodo.17085737 (https://doi.org/10.5281/zenodo.17085737).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 30 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 3 authors, 5 keywords, 1 funder, 83 references.

Cite

This paper

Haberland, S., Ruge, H., & Frimmel, H. (2026). Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks. Imaging neuroscience (Cambridge, Mass.), 4, IMAG.a.1142. https://doi.org/10.1162/imag.a.1142

BibTeX

@article{haberland2026encoding,
author = {Haberland, Sabine and Ruge, Hannes and Frimmel, Holger},
title = {{Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks}},
journal = {Imaging neuroscience (Cambridge, Mass.)},
year = {2026},
month = mar,
volume = {4},
pages = {IMAG.a.1142},
publisher = {MIT Press},
issn = {2837-6056},
doi = {10.1162/imag.a.1142},
url = {https://doi.org/10.1162/imag.a.1142},
pmid = {41852943},
pmcid = {PMC12994000}
}

RIS

TY - JOUR
AU - Haberland, Sabine
AU - Ruge, Hannes
AU - Frimmel, Holger
TI - Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks
T2 - Imaging neuroscience (Cambridge, Mass.)
J2 - Imaging Neurosci (Camb)
PY - 2026
DA - 2026/03/16
VL - 4
SP - IMAG.a.1142
SN - 2837-6056
PB - MIT Press
DO - 10.1162/imag.a.1142
UR - https://doi.org/10.1162/imag.a.1142
LA - en
ER -

CSL-JSON

{
"id": "10.1162/imag.a.1142",
"type": "article-journal",
"title": "Encoding neural representations of time-continuous stimulus-response transformations in the human brain with advanced deep neural networks",
"container-title": "Imaging neuroscience (Cambridge, Mass.)",
"author": [
{
"family": "Haberland",
"given": "Sabine"
},
{
"family": "Ruge",
"given": "Hannes"
},
{
"family": "Frimmel",
"given": "Holger"
}
],
"container-title-short": "Imaging Neurosci (Camb)",
"volume": "4",
"page": "IMAG.a.1142",
"DOI": "10.1162/imag.a.1142",
"PMID": "41852943",
"PMCID": "PMC12994000",
"ISSN": "2837-6056",
"publisher": "MIT Press",
"URL": "https://doi.org/10.1162/imag.a.1142",
"language": "en",
"issued": {
"date-parts": [
[
2026,
3,
16
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-76098-y [code]
A single computational objective can produce specialization of streams in visual cortex.
Journal: Nature communications
In common: scikit-image, Pillow, PyTorch, 2 other tools, 6 references
[2] doi:10.1038/s42003-026-10169-0 [code]
Shared representations in brains and models reveal a two-route cortical organization during scene perception.
Journal: Communications biology
In common: Pillow, PyTorch, NumPy, 7 references
[3] doi:10.1371/journal.pcbi.1014656 [code]
Contrastive learning to fine-tune feature extraction models for the visual cortex.
Journal: PLoS computational biology
In common: Pillow, PyTorch, NumPy, fMRI, systems, 5 references
[4] doi:10.7554/elife.107933 [code]
Modality-agnostic decoding of vision and language from fMRI.
Journal: eLife
In common: SPM, OpenCV, scikit-image, 3 other tools, fMRI, 2 references
[5] doi:10.1038/s41597-026-07248-6 [code]
A large-scale fMRI dataset for vision-language semantic association.
Journal: Scientific data
In common: OpenCV, scikit-image, Pillow, 2 other tools, fMRI, 3 references
[6] doi:10.1162/imag.a.1207 [code]
Investigating the temporal dynamics and modeling of mid-level feature representations in humans.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: OpenCV, Pillow, PyTorch, 1 other tool, 4 references
[7] doi:10.1038/s42003-026-10957-8 [code]
Brain defence by the extracellular matrix protein Cochlin.
Journal: Communications biology
In common: TensorFlow, SPM, OpenCV, 5 other tools
[8] doi:10.1007/s12021-026-09803-3 [code]
NeuroFusion: A Unified Framework for Generalized Visual Stimulus Decoding from fMRI Across Datasets and Subjects.
Journal: Neuroinformatics
In common: scikit-image, Pillow, PyTorch, 1 other tool, fMRI, 3 references
[9] doi:10.1162/imag.a.1286 [code]
Behavioral imitation with artificial neural networks leads to personalized models of brain dynamics during videogame play.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: Pillow, PyTorch, NumPy, fMRI, 4 references
[10] doi:10.3390/s26175501 [code]
Evaluation of a Hybrid Neural-Polynomial Deep Q-Network for Switching-Aware Spectrum Selection in a Controlled Radio-Frequency Measurement-Replay Testbed.
Journal: Sensors (Basel, Switzerland)
In common: TensorFlow, Pillow, PyTorch, 1 other tool, 3 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.