OSCR

Neural Tracking of Sustained Attention, Attention Switching, and Natural Conversation in Audiovisual Environments Using Wearable EEG.

Code ↔ Paper

5 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 5 matches
  1. [1] § Methods and Materials › Data Preprocessing ↔ helpers.py, lines 292–343 · score 0.84 · half wave rectified, frequency bands, acoustic onsets, resolution, GT, edge
  2. [2] § Methods and Materials › Data Preprocessing ↔ preprocessing.ipynb, lines 85–142 · score 0.74 · 1–20 Hz, 0.1–40 Hz, ICA components, cardiac, muscle, removal
  3. [3] § Methods and Materials › Data Analysis › Correlation Metrics and Optimal Lag Analysis ↔ preprocessing.ipynb, lines 376–395 · score 0.59 · cross validation, lag windows, 600 ms, acoustic envelope, folds, forward model
  4. [4] § Methods and Materials › Data Analysis › Correlation Metrics and Optimal Lag Analysis ↔ runTRFs.ipynb, lines 122–260 · score 0.58 · sliding window, cross validation, overlap, lags, 600 ms, optimal
  5. [5] § Results › Neural Data Analysis ↔ plotting.py, lines 596–654 · score 0.56 · confidence interval, Optimal lag, Forward model, cEEGrid, correlation, backward

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 450 lines · 18 KB · MIT · 2 matches

  1. # %% [markdown]
  2. # # Make predictors
  3. #
  4. # This section computes the acoustic predictor variables used as regressors in the TRF models. The pipeline runs in two steps:
  5. #
  6. # 1. **Gammatone spectrograms** — each stimulus audio file (`.wav` / `.mp3`) is passed through a gammatone filter bank (80–15 000 Hz, 128 channels, 1 ms time step) using `eelbrain.gammatone_bank`. The resulting high-resolution spectrograms are cached as `.pickle` files in `stimuli/`.
  7. #
  8. # 2. **Predictor derivatives** — from each spectrogram a set of standard predictor variants is derived and saved to `predictors/`:
  9. # - `~gammatone-1` — log-compressed temporal envelope (summed across all frequency channels → 1 band)
  10. # - `~gammatone-on-1` — acoustic onset envelope (edge-detector applied to the log spectrogram, 1 band)
  11. # - `~gammatone-8` / `~gammatone-on-8` — log envelope / onset binned into 8 frequency bands
  12. # - `~gammatone-lin-8` / `~gammatone-pow-8` — linear-scale and power-law ($x^{0.6}$) spectrograms at 8 bands
  13. # - `~gammatone-onDer-1` — half-wave rectified temporal derivative (1 band)
  14. #
  15. # Both steps skip files that have already been processed, so the cells are safe to re-run.
  17. # %%
  18. from pathlib import Path
  19. import eelbrain as eel
  20. import numpy as np
  21. from pydub import AudioSegment
  22. from helpers import make_gammatone, make_gt_predictor
  24. ROOT = Path.cwd()
  25. STIMULUS_DIR = ROOT / 'stimuli'
  26. PREDICTOR_DIR = ROOT / 'predictors'
  27. PREDICTOR_DIR.mkdir(exist_ok=True)
  29. # %% [markdown]
  30. # ## Make gammatone files
  31. #
  32. # Generates high-resolution gammatone spectrograms from stimulus audio files using `eelbrain.gammatone_bank` (80–15000 Hz, 128 channels, 1 ms time step) and saves them as `.pickle` files. Adapted from [`make_gammatone.py`](https://github.com/Eelbrain/Alice/blob/main/predictors/make_gammatone.py) in the Alice dataset.
  34. # %%
  35. filenames = [f.name for f in STIMULUS_DIR.iterdir() if f.is_file()]
  36. for ii in filenames:
  37. i = ii[:-4]
  38. filepath_wav = STIMULUS_DIR / f'{i}.wav'
  39. filepath_mp3 = STIMULUS_DIR / f'{i}.mp3'
  41. if filepath_wav.exists() and filepath_wav.stat().st_size > 0:
  42. print(f"Read {filepath_wav}")
  43. wav = eel.load.wav(filepath_wav)
  44. elif filepath_mp3.exists():
  45. print(f"Read {filepath_mp3}")
  47. audio = AudioSegment.from_mp3(filepath_mp3)
  48. samples = np.array(audio.get_array_of_samples())
  49. samples = samples.astype(np.float32) / (2**15) # normalize 16-bit
  50. fs = audio.frame_rate
  52. # convert to eelbrain NDVar
  53. time = eel.UTS(0, 1/fs, samples.shape[0])
  54. wav = eel.NDVar(samples, (time,))
  56. else:
  57. print(f"File {i} is missing or empty...")
  58. continue
  60. make_gammatone(wav,i)
  62. # %% [markdown]
  63. # ## Make gammatone predictors
  64. #
  65. # Derives predictor variables from the gammatone spectrograms created above. Applies log scaling and an onset filter, then sums across frequency bands to produce 1-band and 8-band envelope predictors saved as `.pickle` files. Adapted from [`make_gammatone_predictors.py`](https://github.com/Eelbrain/Alice/blob/main/predictors/make_gammatone_predictors.py) in the Alice dataset.
  67. # %%
  68. filenames = [f.name for f in STIMULUS_DIR.iterdir() if f.is_file() and (f.name.endswith('.pickle'))]
  69. name = '-gammatone'
  71. conv_names = ['left', 'right', 'mean']
  73. nfiles = len(filenames)
  75. for nn, ii in enumerate(filenames):
  76. print(f'file: {nn+1}/{nfiles} {filenames[nn]}')
  78. i = ii[:-7]
  79. j = i[:-len(name)]
  80. dst = PREDICTOR_DIR / f'{j}~gammatone-1.pickle'
  81. if dst.exists():
  82. continue
  83. make_gt_predictor(i,j)
  85. # %% [markdown]
  86. # # Preprocess EEG
  87. #
  88. # This section loads the raw EEG recordings and prepares them for TRF analysis. The pipeline covers five steps that must be run in order:
  89. #
  90. # 1. **Convert XDF → FIF** — reads each raw `.xdf` recording (one per subject and sensor array) and saves it as an MNE `.fif` file under `data/sub-<id>/eeg/`. This format is required by the eelbrain TRF pipeline. Run once per subject.
  91. #
  92. # 2. **Initiate TRF pipeline** — imports libraries, resolves data paths, patches the MNE-BIDS key validator to allow compound acquisition names (e.g. `scalpSustA`), and discovers all subject folders under `data/`.
  93. #
  94. # 3. **Make ICA** — fits an ICA decomposition (20 components for scalp, 10 for cEEGrid) on the 1–40 Hz band-pass filtered data and saves the solution to `data/derivatives/ica/`. Run once per subject.
  95. #
  96. # 4. **Select ICA components to exclude** — opens an interactive GUI to manually inspect and mark artifactual ICA components (eye blinks, muscle noise, cardiac, line noise) for removal. Run once per subject after ICA fitting.
  97. #
  98. # 5. **Align EEG and predictors** — applies the accepted ICA solution to the 0.1–40 Hz filtered data, re-filters to 1–20 Hz, extracts stimulus-locked epochs, resamples to 50 Hz, z-scores each channel, and time-aligns the corresponding gammatone predictor variables. For the switA condition, epochs and predictors are additionally split at the attention-switch boundaries (t = 35 s and t = 125 s). The final EEG + predictor bundles are saved as `.pickle` files under `data/derivatives/preprocessed/`.
  100. # %% [markdown]
  101. # ## Make .xdf to .fif files
  102. #
  103. # The cell:
  104. # 1. Converts the raw `.xdf` recording to MNE `.fif` format (via `save_xdf_as_fif`) if not already done. The correct data structure need a .fif file to initiate eelbrain TRF pipeline.
  106. # %%
  107. from eelbrain import *
  108. from helpers import save_xdf_as_fif
  109. from pathlib import Path
  110. import re
  112. ROOT = Path.cwd()
  113. DATA_ROOT = Path.cwd() / 'data'
  115. subjects = [
  116. p.name.removeprefix("sub-")
  117. for p in DATA_ROOT.iterdir()
  118. if p.is_dir() and p.name.startswith("sub-")
  119. ]
  120. print(f'Subjects: {subjects}')
  122. exclude_subs = []
  123. filt = [1, 20]
  124. aquisitions = ['scalpSustA', 'ceegridSustA', 'scalpSwitA', 'ceegridSwitA', 'scalpConvA', 'ceegridConvA']
  126. for subject in subjects:
  127. print(f'sub-{subject}')
  128. for acq_combined in aquisitions:
  129. acq_type = re.match(r'^(scalp|ceegrid)', acq_combined).group(1)
  130. task_cap = re.sub(r'^(scalp|ceegrid)', '', acq_combined)
  131. task = task_cap[0].lower() + task_cap[1:]
  133. dst1 = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_task-{task}_eeg.xdf'
  134. if not dst1.exists():
  135. print(f' {acq_combined}: xdf file not found, skipping')
  136. continue
  138. # No-task file saved by save_xdf_as_fif (e.g. sub-99_acq-scalpSustA_eeg.fif)
  139. dst2 = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_acq-{acq_combined}_eeg.fif'
  140. if not dst2.exists():
  141. print(f' {acq_combined}: making fif files...')
  142. save_xdf_as_fif(task, subject)
  144. # %% [markdown]
  145. # ## Initiate TRF pipeline
  146. #
  147. # The TRF pipline can only be initiated correctly if:
  148. #
  149. # DATA_ROOT / f'sub-{id}' / 'eeg' / 'sub-{id}_acq-{acq}_eeg.fif'
  150. #
  151. # files exist (for example: sub-99_acq-scalpSustA_eeg.fif). To make these .fif files from raw .xdf files, run previous cell.
  152. #
  153. # NOTE (experiment.py):
  154. #
  155. # tasks = tasks = ["sustA"] #, "switA", "convA"]
  156. #
  157. # since we only provide data for sustA.
  159. # %%
  160. from eelbrain import *
  161. from experiment import mobEEG_e, PARAMETERS
  162. from helpers import save_xdf_as_fif, make_gammatone, make_gt_predictor
  163. import eelbrain as eel
  164. import numpy as np
  165. from scipy.signal import resample
  166. import shutil
  169. # Allow underscores in BIDS 'acq' field (e.g. 'sustA_scalp') — eelbrain passes
  170. # the acquisition value through to mne_bids which otherwise rejects underscores.
  171. import mne_bids.utils as _mbu
  172. _orig_check = _mbu._check_key_val
  173. def _patched_check(key, val):
  174. if key == 'acq':
  175. return key, val
  176. return _orig_check(key, val)
  177. _mbu._check_key_val = _patched_check
  179. SAVE_DIR = ROOT / 'results'
  180. SAVE_DIR.mkdir(exist_ok=True)
  183. # %% [markdown]
  184. # ## Make ICA
  185. #
  186. # Fits an Independent Component Analysis (ICA) decomposition for each subject and sensor array (scalp / cEEGrid) across all three task conditions (sustA, switA, convA). **Only needs to be run once per subject.**
  187. #
  188. # The cell:
  189. # 1. Creates a BIDS-compatible copy of the `.fif` file with the `task-` field in the filename, as required by eelbrain's pipeline.
  190. # 2. Calls `mobEEG_e.make_ica()` which band-pass filters the data (1–40 Hz) and fits an ICA solution — 20 components for scalp, 10 for cEEGrid — saving the result to `data/derivatives/ica/`.
  191. #
  192. # Subjects or acquisitions for which an ICA file already exists are silently skipped.
  194. # %%
  195. ica_dir = DATA_ROOT / 'derivatives' / 'ica'
  196. ica_dir.mkdir(parents=True, exist_ok=True)
  198. for subject in subjects:
  199. print(f'sub-{subject}')
  200. for acq_combined in aquisitions:
  201. acq_type = re.match(r'^(scalp|ceegrid)', acq_combined).group(1)
  202. task_cap = re.sub(r'^(scalp|ceegrid)', '', acq_combined)
  203. task = task_cap[0].lower() + task_cap[1:]
  205. dst1 = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_task-{task}_eeg.xdf'
  206. if not dst1.exists():
  207. print(f' {acq_combined}: xdf file not found, skipping')
  208. continue
  210. # No-task file saved by save_xdf_as_fif (e.g. sub-99_acq-scalpSustA_eeg.fif)
  211. dst2 = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_acq-{acq_combined}_eeg.fif'
  212. if not dst2.exists():
  213. print(f' {dst2} is missing... Run cell "make .xdf to .fif"')
  214. continue
  216. # Eelbrain's BIDS path builder requires task- in the filename; create a copy
  217. dst2_bids = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_task-{task}_acq-{acq_combined}_eeg.fif'
  218. if not dst2_bids.exists():
  219. shutil.copy2(dst2, dst2_bids)
  221. ica_file = ica_dir / f'sub-{subject}_acq-{acq_combined}_eeg_raw-ica_ica.fif'
  222. if ica_file.exists():
  223. print(f' {acq_combined}: ICA already exists, skipping')
  224. continue
  226. try:
  227. mobEEG_e.set(subject=subject, raw=f'{task}_ica_{acq_type}', acquisition=acq_combined)
  228. mobEEG_e.make_ica()
  229. print(f' Saved: {ica_file.name}')
  230. except FileNotFoundError as err:
  232. print(f' {acq_combined}: file not found — {err}')
  234. # %% [markdown]
  235. # ## Select ICA components to exclude
  236. #
  237. # Opens the eelbrain ICA selection GUI for each subject and sensor array so that artifactual components (eye blinks, muscle noise, cardiac, line noise) can be manually identified and marked for removal. **Only needs to be run once per subject**, after ICA has been fitted above.
  238. #
  239. # The cell iterates over all subjects and acquisitions and calls `mobEEG_e.make_ica_selection()`, which renders each component as a time-series, scalp topography, and power spectrum. Click components to toggle their exclusion status and close the window to save the selection.
  240. #
  241. # > **Note:** The GUI runs inside the notebook kernel's event loop. If the window freezes or does not respond, use `select_ica_gui.py` instead, which launches the GUI in a separate process:
  242. # > ```python
  243. # > import subprocess, sys
  244. # > subprocess.run([sys.executable, 'select_ica_gui.py', subject, condition, acq_type])
  245. # > ```
  246. # > The selected components are stored back in the condition-specific ICA file under `data/derivatives/ica/` and applied automatically during the **Align EEG and predictors** step below.
  248. # %%
  249. ica_dir = DATA_ROOT / 'derivatives' / 'ica'
  251. for subject in subjects:
  252. print(f'sub-{subject}')
  253. for acq_combined in aquisitions:
  254. acq_type = re.match(r'^(scalp|ceegrid)', acq_combined).group(1)
  255. task_cap = re.sub(r'^(scalp|ceegrid)', '', acq_combined)
  256. task = task_cap[0].lower() + task_cap[1:]
  258. ica_file = ica_dir / f'sub-{subject}_acq-{acq_combined}_eeg_raw-ica_ica.fif'
  259. if not ica_file.exists():
  260. print(f' {acq_combined}: ICA not found, skipping')
  261. continue
  264. mobEEG_e.set(subject=subject, raw=f'{task}_ica_{acq_type}', acquisition=acq_combined)
  266. mobEEG_e.make_ica_selection()
  267. print(f' {acq_combined}: done')
  269. # %% [markdown]
  270. # # Align EEG and predictors
  272. # %%
  273. SAVE_DIR = DATA_ROOT / 'derivatives' / 'preprocessed'
  274. SAVE_DIR.mkdir(exist_ok=True)
  277. fieldnames = ['eeg','predictors','microphones']
  278. attentions = ['attended','ignored','null']
  280. # Combined condition_acquisition names
  281. typeOfRegressors = ['~gammatone-1','~gammatone-on-1']
  282. micRegressors = ['gt_log1','gt_on1']
  284. predictor_names = ['env', 'onset']
  286. data = {
  287. 'eeg': [],
  288. 'info': {},
  289. 'microphones': {},
  290. 'predictors': {
  291. attention: {
  292. pred: []
  293. for pred in predictor_names
  294. }
  295. for attention in attentions
  296. }
  297. }
  299. info = {
  300. 'summary': 'Preprocessed data ready for TRF analysis.',
  301. 'condition': '',
  302. 'acquisition': '',
  303. 'attended': 'Attended speech',
  304. 'ignored': 'Ignored speech',
  305. 'null': 'Trial-shifted predictors as null model',
  306. 'normalization': 'eeg, predictors and mics are z-normalized',
  307. 'filter': filt,
  308. 'fs': 50,
  309. 'cut': 'First and last 1s of each block are cut to avoid edge artifacts from filtering and epoching',
  310. }
  314. info_switA = {
  315. 'summary': 'Preprocessed data ready for TRF analysis. Switches are randomized between the intervals',
  316. 'block1': [0,35],
  317. 'switch1': [35,55],
  318. 'block2': [55,125],
  319. 'switch2': [125,145],
  320. 'block3': [145,178],
  321. 'attended': 'Predictors for attended block1 and block3',
  322. 'ignored': 'Predictors for attended block2',
  323. 'null': 'Trial-shifted predictors as null model',
  324. }
  327. for subject in subjects:
  328. print('---------------------------')
  329. print(f'Subject: {subject}')
  330. out_path = SAVE_DIR / f'sub-{subject}'
  331. out_path.mkdir(parents=True, exist_ok=True)
  333. for acq_combined in aquisitions:
  334. acq_type = re.match(r'^(scalp|ceegrid)', acq_combined).group(1)
  335. task_cap = re.sub(r'^(scalp|ceegrid)', '', acq_combined)
  336. task = task_cap[0].lower() + task_cap[1:]
  337. print(f'Condition: {task}, Acq: {acq_combined}')
  339. info['condition'] = task
  340. info['acquisition'] = acq_combined
  342. if task == 'switA':
  343. info['switchingInfo'] = info_switA
  344. else:
  345. info['switchingInfo'] = 'No switching'
  347. eeg_fif = DATA_ROOT / f'sub-{subject}' / 'eeg' / f'sub-{subject}_acq-{acq_combined}_eeg.fif'
  348. if not eeg_fif.exists():
  349. print(f' {acq_combined}: EEG file not found, skipping')
  350. continue
  352. save_path = out_path / f'sub-{subject}_acq-{acq_combined}.pickle'
  353. if save_path.exists():
  354. continue
  356. mobEEG_e.set(subject=subject, task=task, raw=f'1-20_{task}_{acq_type}', acquisition=acq_combined)
  358. epochs, predictors, predictors_mic, predictors_attended, attended_conv = mobEEG_e.align_epochs_and_predictors(
  359. typeOfRegressors, micRegressors, filt, condition=task
  360. )
  362. eeg_ndvar_all = eel.load.mne.epochs_ndvar(epochs)
  363. data['eeg'] = eeg_ndvar_all
  365. data['predictors']['attended']['env'] = predictors['attended_files']['~gammatone-1'][0]
  366. data['predictors']['attended']['onset'] = predictors['attended_files']['~gammatone-on-1'][0]
  367. data['predictors']['ignored']['env'] = predictors['ignored_files']['~gammatone-1'][0]
  368. data['predictors']['ignored']['onset'] = predictors['ignored_files']['~gammatone-on-1'][0]
  369. data['predictors']['null']['env'] = predictors['shifted_files']['~gammatone-1'][0]
  370. data['predictors']['null']['onset'] = predictors['shifted_files']['~gammatone-on-1'][0]
  372. data['info'] = info
  374. eel.save.pickle(data, save_path)
  376. # %% [markdown]
  377. # # Quick TRF-check
  378. #
  379. # A lightweight sanity check to verify that the preprocessed data for a single subject looks reasonable before running the full group analysis. Loads one subject's `.pickle` file and fits both backward and forward TRF models using `eelbrain.boosting` with L1 error and 9-fold cross-validation.
  380. #
  381. # - **Backward model** (predictor → EEG, lag window −0.6 – 0.2 s) — estimates how well the acoustic envelope predicts the EEG response. Reports the cross-validated correlation *r* for attended, ignored, and null (trial-shifted) predictors. Attended *r* should be clearly higher than null.
  382. # - **Forward model** (EEG → predictor, lag window −0.2 – 0.6 s) — estimates the temporal response function (TRF) as a scalp topography over time. The attended TRF should show clear auditory components (e.g. N1 at ~100 ms); the null TRF should be flat.
  383. #
  384. # Set `acq` at the top of the load cell to switch between sensor arrays and conditions (e.g. `'scalpSustA'`, `'ceegridSwitA'`).
  386. # %%
  387. acq = 'scalpSustA'
  389. tmp = SAVE_DIR / f'sub-{subject}' / f'sub-{subject}_acq-{acq}.pickle'
  390. data = eel.load.unpickle(tmp)
  392. eeg = data['eeg']
  393. attended = data['predictors']['attended']['env']
  394. ignored = data['predictors']['ignored']['env']
  395. null = data['predictors']['null']['env']
  397. # %% [markdown]
  398. # ## Backward models
  400. # %%
  401. TMIN = -0.6
  402. TMAX = 0.2
  403. nPartitions = 9
  405. bw_att = eel.boosting(attended, eeg, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  406. print(f'Attended r: {float(bw_att.r)}')
  408. bw_ign = eel.boosting(ignored, eeg, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  409. print(f'Ignored r: {float(bw_ign.r)}')
  411. bw_null = eel.boosting(null, eeg, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  412. print(f'Null r: {float(bw_null.r)}')
  416. # %% [markdown]
  417. # ## Forward models
  419. # %%
  420. TMIN = -0.2
  421. TMAX = 0.6
  423. fw_att = eel.boosting(eeg, attended, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  425. fw_ign = eel.boosting(eeg, ignored, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  427. fw_null = eel.boosting(eeg, null, TMIN, TMAX, basis=0.05, error='l1', partitions=nPartitions, test=1, partition_results=True)
  430. vmax = float(max(
  431. abs(fw_att.h_scaled.x).max(),
  432. abs(fw_ign.h_scaled.x).max(),
  433. abs(fw_null.h_scaled.x).max()
  434. ))
  436. def clean_spines(p):
  437. for ax in p.axes:
  438. ax.spines['top'].set_visible(False)
  439. ax.spines['right'].set_visible(False)
  440. ax.axvline(0, color='grey', linestyle='--', alpha=0.6)
  441. ax.grid(True, alpha=0.2)
  443. p_att = eel.plot.Butterfly(fw_att.h_scaled, title='Attended TRF', vmax=vmax)
  444. clean_spines(p_att)
  446. p_ign = eel.plot.Butterfly(fw_ign.h_scaled, title='Ignored TRF', vmax=vmax)
  447. clean_spines(p_ign)
  449. p_null = eel.plot.Butterfly(fw_null.h_scaled, title='Null TRF', vmax=vmax)
  450. clean_spines(p_null)

preprocessing.ipynb at commit 4cce465, under MIT · at the source

Overview

Authors: Johanna Wilroth1, Oskar Keding2, Martin A Skoglund1,3, Maria Sandsten2, Martin Enqvist1, Emina Alickovic1,3
ORCID iDs: Oskar Keding, Emina Alickovic
  1. Department of Electrical Engineering, Linköping University, Linköping, Sweden
  2. Centre for Mathematical Sciences, Lund University, Lund, Sweden
  3. Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
Institutions: Linköping University (Sweden); Lund University (Sweden); Oticon (Denmark) (Denmark)
Journal: The European journal of neuroscience, volume 63, issue 9, article e70538
Dates: received 30 December 2025; accepted 27 April 2026; published online 8 May 2026; in print May 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1111/ejn.70538 · PMID 42104679 · PMCID PMC13156524 · OpenAlex W7160713770
Open access: hybrid, a free copy (OpenAlex)
Status: code verified
Categories: EEG (modality), other (modality), human (organism), cognitive (subfield)
Methods: Spectral & time-frequency, Preprocessing, Connectivity, Smoothing, state filtering, decompositions, Machine learning, Physiology & signal measures
Keywords: audiovisual stimuli, auditory attention decoding, conversation, neural speech tracking, wearable EEG
MeSH: Attention*, Auditory Perception*, Brain*, Electroencephalography*, Speech Perception*, Visual Perception*, Acoustic Stimulation, Adult, Female, Humans, Male, Photic Stimulation, Wearable Electronic Devices, Young Adult (* major topic)
Topic: Multisensory perception and integration (Experimental and Cognitive Psychology, Psychology), according to OpenAlex
Funding: ELLIIT Research Programme, Sweden (C04)
Citations: cited by 1 paper (Europe PMC); 63 references in the paper

Abstract

Everyday communication is dynamic and multisensory, often involving shifting attention, overlapping speech, and visual cues. Yet, most neural attention tracking studies are still limited to highly controlled lab settings, using clean, often audio‐only stimuli and requiring sustained attention to a single talker. This work addresses that gap by introducing a novel dataset from 24 normal‐hearing participants. We used a wearable electroencephalography (EEG) system (44 scalp electrodes and 20 cEEGrid electrodes) in an audiovisual (AV) paradigm with three conditions: sustained attention to a single talker in a two‐talker environment, attention switching between two talkers, and unscripted two‐talker conversations with a competing single talker. Analysis included temporal response functions (TRFs) modeling, optimal lag analysis, selective attention classification with decision windows ranging from 1.1 to 35 s, and comparisons of TRFs for attention to AV conversations versus side audio‐only talkers. Key findings show significant differences in the attention‐related P2 peak between attended and ignored speech across conditions for scalp EEG. Interestingly, our results revealed strong cross‐condition generalization, with models trained in one condition maintaining good performance when evaluated on the other two. No significant change in performance between switching and sustained attention suggests robustness for attention switches. Optimal lag analysis revealed a narrower peak for conversation compared to single‐talker AV stimuli, reflecting the additional complexity of multi‐talker processing. Classification of selective attention was consistently above chance (55%–70% accuracy) for scalp EEG, whereas cEEGrid data yielded lower correlations, highlighting the need for further methodological improvements. These results demonstrate that wearable EEG can reliably track selective attention in dynamic, multisensory listening scenarios and provide guidance for designing future AV paradigms and real‐world attention tracking applications.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 5 matches between paragraphs and lines of code.

JohannaWil/audiovisual-attention-eeg-trf

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 4cce46580a14e1feea78da884c77fca8a2c5c6e1, 11 May 2026
Languages: Python (4), Jupyter (3)
Size: 46 files, 7 scripts
Software Heritage: not archived
Found in: “Data Availability Statement”
Holds: README, license file, 3 notebooks
Not found: CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (5 files), Matplotlib (3 files), SciPy (3 files), MNE-Python (2 files), pandas (2 files), seaborn (2 files), MNE-BIDS (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
9 files
At the source: github.com/JohannaWil/audiovisual-attention-eeg-trf/

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 7 scripts, each with its path and the digest of its content;
  • 5 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

The data underlying this study are subject to ethical and legal restrictions. Participant consent did not explicitly include permission for public data sharing, and the dataset falls under the scope of the EU General Data Protection Regulation (GDPR). For these reasons, the data are not publicly available. Data may be made available upon reasonable request, subject to institutional and ethical approval and appropriate data‐sharing agreements. The code is available at github.com/JohannaWil/audiovisual‐attention‐eeg‐trf (https://github.com/JohannaWil/audiovisual-attention-eeg-trf/).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

  • Publisher: — → Wiley

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 6 authors, 5 keywords, 14 MeSH terms, 1 funder, 52 references.

Cite

This paper

Wilroth, J., Keding, O., Skoglund, M. A., Sandsten, M., Enqvist, M., & Alickovic, E. (2026). Neural Tracking of Sustained Attention, Attention Switching, and Natural Conversation in Audiovisual Environments Using Wearable EEG. The European journal of neuroscience, 63(9), e70538. https://doi.org/10.1111/ejn.70538

BibTeX

@article{wilroth2026neural,
author = {Wilroth, Johanna and Keding, Oskar and Skoglund, Martin A and Sandsten, Maria and Enqvist, Martin and Alickovic, Emina},
title = {{Neural Tracking of Sustained Attention, Attention Switching, and Natural Conversation in Audiovisual Environments Using Wearable EEG}},
journal = {The European journal of neuroscience},
year = {2026},
month = may,
volume = {63},
number = {9},
pages = {e70538},
publisher = {Wiley},
issn = {0953-816X},
doi = {10.1111/ejn.70538},
url = {https://doi.org/10.1111/ejn.70538},
pmid = {42104679},
pmcid = {PMC13156524}
}

RIS

TY - JOUR
AU - Wilroth, Johanna
AU - Keding, Oskar
AU - Skoglund, Martin A
AU - Sandsten, Maria
AU - Enqvist, Martin
AU - Alickovic, Emina
TI - Neural Tracking of Sustained Attention, Attention Switching, and Natural Conversation in Audiovisual Environments Using Wearable EEG
T2 - The European journal of neuroscience
J2 - Eur J Neurosci
PY - 2026
DA - 2026/05/01
VL - 63
IS - 9
SP - e70538
SN - 0953-816X
PB - Wiley
DO - 10.1111/ejn.70538
UR - https://doi.org/10.1111/ejn.70538
LA - en
ER -

CSL-JSON

{
"id": "10.1111/ejn.70538",
"type": "article-journal",
"title": "Neural Tracking of Sustained Attention, Attention Switching, and Natural Conversation in Audiovisual Environments Using Wearable EEG",
"container-title": "The European journal of neuroscience",
"author": [
{
"family": "Wilroth",
"given": "Johanna"
},
{
"family": "Keding",
"given": "Oskar"
},
{
"family": "Skoglund",
"given": "Martin A"
},
{
"family": "Sandsten",
"given": "Maria"
},
{
"family": "Enqvist",
"given": "Martin"
},
{
"family": "Alickovic",
"given": "Emina"
}
],
"container-title-short": "Eur J Neurosci",
"volume": "63",
"issue": "9",
"page": "e70538",
"DOI": "10.1111/ejn.70538",
"PMID": "42104679",
"PMCID": "PMC13156524",
"ISSN": "0953-816X",
"publisher": "Wiley",
"URL": "https://doi.org/10.1111/ejn.70538",
"language": "en",
"issued": {
"date-parts": [
[
2026,
5,
1
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.7554/elife.100605 [code]
Age-related changes in ‘cortical’ 1/f dynamics are linked to cardiac activity
Journal: —
In common: MNE-BIDS, MNE-Python, seaborn, 4 other tools, other, 2 references
[2] doi:10.1038/s41597-025-06397-4 [code]
MEG-SCANS - A comprehensive magnetoencephalography speech dataset with Stories, Chirps and Noisy Sentences
Journal: —
In common: MNE-Python, pandas, SciPy, 2 other tools, 5 references
[3] doi:10.1371/journal.pbio.3003876 [code]
Competing speech streams are simultaneously represented in the human cortex during attention switching.
Journal: PLoS biology
In common: EEG, cognitive, 6 references
[4] doi:10.1038/s42003-026-10394-7 [code]
Cognitive load weakens neural speech tracking without altering response timing.
Journal: Communications biology
In common: MNE-Python, seaborn, pandas, 3 other tools, EEG, cognitive, 3 references
[5] doi:10.1038/s41598-026-55371-6 [code]
Decoding selective auditory attention to musical elements in ecologically valid music listening.
Journal: Scientific reports
In common: MNE-Python, pandas, NumPy, EEG, cognitive, 4 references
[6] doi:10.1097/j.pain.0000000000004044 [code]
No effect of rhythmic visual stimulation on experimental pain perception.
Journal: Pain
In common: MNE-BIDS, MNE-Python, seaborn, 4 other tools, EEG, cognitive, 1 reference
[7] doi:10.1093/cercor/bhag113 [code]
Long-term reliability and stability of parameterized resting state EEG: evidence from a five-year follow-up.
Journal: Cerebral cortex (New York, N.Y. : 1991)
In common: MNE-BIDS, MNE-Python, seaborn, 4 other tools, EEG, 1 reference
[8] doi:10.1038/s41597-025-05174-7 [code]
A large-scale MEG and EEG dataset for object recognition in naturalistic scenes
Journal: —
In common: MNE-BIDS, MNE-Python, seaborn, 4 other tools, EEG, 1 reference
[9] doi:10.1093/nc/niag029 [code]
A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception.
Journal: Neuroscience of consciousness
In common: MNE-BIDS, MNE-Python, seaborn, 4 other tools, cognitive, 1 reference
[10] doi:10.1038/s41467-026-75240-0 [code]
Dynamic acoustic-to-categorical representations of phonemes and prosody along ventral and dorsal speech streams.
Journal: Nature communications
In common: MNE-Python, seaborn, pandas, 3 other tools, cognitive, 2 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.