OSCR

Compositional Complexity in Text and Images.

Code ↔ Paper

12 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 12 matches
  1. [1] § MATERIALS AND METHODS › Experimental Procedure ↔ src/compositionality_study/experiment/mri/convert_hf_ds_to_local_files.py, lines 214–270 · score 0.82 · inter stimulus interval, blank trials, randomized, ISI, duration, split
  2. [2] § MATERIALS AND METHODS › Experimental Procedure ↔ src/compositionality_study/experiment/mri/generate_psychopy_files.py, lines 206–354 · score 0.80 · inter stimulus interval, frame rate, practice stimuli, PsychoPy, trigger, duration
  3. [3] § MATERIALS AND METHODS › Stimuli › Stimulus selection ↔ src/compositionality_study/data/select_stimuli.py, lines 134–208 · score 0.75 · propensity score matching, PsmPy, duplications, subset, filtered, graph
  4. [4] § MATERIALS AND METHODS › Image Preprocessing ↔ src/compositionality_study/models/univariate_analysis.py, lines 61–167 · score 0.72 · motion regressors, drift model, cosine, GLMs, smoothing, scans
  5. [5] § MATERIALS AND METHODS › Statistical Analyses ↔ src/compositionality_study/models/mvpa_decoder.py, lines 194–285 · score 0.68 · modality decoding, radius, stratified, fold, searchlight, linear
  6. [6] § MATERIALS AND METHODS › Stimuli › Textual compositional complexity ↔ src/compositionality_study/utils.py, lines 279–325 · score 0.61 · dependency parsing, graph depth, AMR graph, trees, edges, nodes
  7. [7] § MATERIALS AND METHODS › Stimuli › Textual compositional complexity ↔ src/compositionality_study/visualization/visualize_selected_stimuli.py, lines 227–307 · score 0.61 · dependency parsing, graph depth, AMR graph, attributes, trees, model
  8. [8] § MATERIALS AND METHODS › Image Preprocessing ↔ src/compositionality_study/models/post_hoc.py, lines 58–123 · score 0.57 · drift model, cosine, motion, GLMs, smoothing, confounds
  9. [9] § MATERIALS AND METHODS › Stimuli › Textual compositional complexity ↔ src/compositionality_study/utils.py, lines 279–325 · score 0.55 · dependency parsing, AMR graph, rooted, trees, edges, sentence
  10. [10] § MATERIALS AND METHODS › Stimuli › Visual compositional complexity ↔ src/compositionality_study/utils.py, lines 227–257 · score 0.53 · graph depth, AMR graphs, longest, shortest, edges, nodes
  11. [11] § RESULTS › Multivariate Pattern Analysis of Compositional Complexity ↔ src/compositionality_study/models/estimate_betas.py, lines 82–128 · score 0.51 · extra confounds, aspect ratio, beta, nodes, model, AMR
  12. [12] § RESULTS › Univariate Analysis of Compositional Complexity ↔ src/compositionality_study/models/univariate_analysis.py, lines 536–594 · score 0.50 · functional localization mask, conjunction, row, confounds, frame, Univariate

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 510 lines · 18 KB · MIT · 3 matches

  1. """Utils for the compositionality project."""
  3. from typing import Dict, Iterable, List, Optional, Tuple, Union
  4. from pathlib import Path
  6. import amrlib
  7. import networkx as nx
  8. import nibabel as nib
  9. import numpy as np
  10. import pandas as pd
  11. import penman
  12. from datasets import load_dataset
  13. from loguru import logger
  14. from nilearn import image, plotting
  15. from nilearn.glm.thresholding import threshold_stats_img
  16. from penman.exceptions import DecodeError
  17. from PIL import Image, ImageOps
  18. from spacy.language import Language # type: ignore
  19. from spacy.tokens import Doc, Token
  21. from compositionality_study.constants import HF_DATASET_NAME
  23. # Set up the spacy amrlib extension
  24. amrlib.setup_spacy_extension()
  26. _COCO_DF = None
  28. DEFAULT_VOXEL_P = 0.001
  29. DEFAULT_CLUSTER_THRESHOLD = 50
  32. def get_coco_df() -> pd.DataFrame:
  33. """Load and cache the COCO dataset."""
  34. global _COCO_DF
  35. if _COCO_DF is None:
  36. ds = load_dataset(HF_DATASET_NAME, split="train")
  37. df = ds.to_pandas()
  38. if isinstance(df, pd.DataFrame):
  39. _COCO_DF = df
  40. else:
  41. raise ValueError("Expected DataFrame")
  42. return _COCO_DF # type: ignore
  45. def get_stimulus_features_lookup(coco_df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame]:
  46. """Create optimized lookup tables for text and image features."""
  47. txt_df = (
  48. coco_df.drop_duplicates("sentences_raw").set_index("sentences_raw")
  49. if "sentences_raw" in coco_df.columns
  50. else pd.DataFrame()
  51. )
  52. img_df = (
  53. coco_df.drop_duplicates("cocoid").set_index("cocoid")
  54. if "cocoid" in coco_df.columns
  55. else pd.DataFrame()
  56. )
  57. return txt_df, img_df
  60. def get_stimulus_data(
  61. modality: Optional[str],
  62. stimulus: Optional[str],
  63. cocoid: Union[str, float, int, None],
  64. txt_df: pd.DataFrame,
  65. img_df: pd.DataFrame,
  66. ) -> Union[pd.Series, None]: # type: ignore
  67. """Retrieve features for a single stimulus event."""
  68. if modality == "text" and stimulus in txt_df.index:
  69. res = txt_df.loc[stimulus]
  70. return res if isinstance(res, pd.Series) else res.iloc[0]
  71. elif modality == "image":
  72. if pd.notna(cocoid):
  73. if cocoid in img_df.index:
  74. res = img_df.loc[cocoid]
  75. return res if isinstance(res, pd.Series) else res.iloc[0]
  76. try:
  77. # Handle potential float/string mismatches
  78. int_cid = int(cocoid) # type: ignore
  79. if int_cid in img_df.index:
  80. res = img_df.loc[int_cid]
  81. return res if isinstance(res, pd.Series) else res.iloc[0]
  82. except (ValueError, TypeError):
  83. pass
  84. return None
  87. def derive_run_id(bold_file: Path) -> str:
  88. """Strip space/desc tokens to get the BIDS run identifier."""
  89. stem = bold_file.name.replace(".nii.gz", "").replace(".nii", "")
  90. parts: List[str] = []
  92. for token in stem.split("_"):
  93. if token.startswith("space-") or token.startswith("desc-"):
  94. break
  95. parts.append(token)
  97. return "_".join(parts)
  100. def load_events(events_tsv: Path) -> pd.DataFrame:
  101. """Load events and normalize the condition column name."""
  102. events = pd.read_csv(events_tsv, sep="\t")
  104. modality = events["modality"].astype(str).str.strip().str.lower()
  105. complexity = events["complexity"].astype(str).str.strip().str.lower()
  106. events["trial_type"] = modality + "_" + complexity
  108. if "trial_type" in events.columns:
  109. events = events.dropna(subset=["trial_type"])
  110. events["trial_type"] = events["trial_type"].astype(str)
  112. return events
  115. def find_gm_mask(fmriprep_dir: Path, subject: str) -> Path:
  116. """Pick the GM probseg (assuming MNI152NLin2009cAsym space)."""
  118. anat_dir = fmriprep_dir / f"sub-{subject}" / "ses-01" / "anat"
  119. candidates = sorted(anat_dir.glob("*space-MNI152NLin2009cAsym*_label-GM_probseg.nii.gz"))
  120. return candidates[0]
  123. def collect_runs(
  124. fmriprep_dir: Path, bids_dir: Path, subject: str, max_runs: int | None = None
  125. ) -> List[Tuple[Path, Path, Path]]:
  126. """Pair each preprocessed BOLD run with matching events and confounds."""
  128. bold_files = sorted((fmriprep_dir / f"sub-{subject}").glob("**/*_desc-preproc_bold.nii.gz"))
  129. runs: List[Tuple[Path, Path, Path]] = []
  131. if max_runs is not None:
  132. logger.info(f"Debugging: limiting to {max_runs} runs for subject {subject}")
  133. bold_files = bold_files[:max_runs]
  135. for bold_file in bold_files:
  136. run_id = derive_run_id(bold_file)
  137. rel_dir = bold_file.parent.relative_to(fmriprep_dir / f"sub-{subject}")
  139. events_file = bids_dir / f"sub-{subject}" / rel_dir / f"{run_id}_events.tsv"
  140. confounds_file = bold_file.parent / f"{run_id}_desc-confounds_timeseries.tsv"
  142. runs.append((bold_file, events_file, confounds_file))
  144. return runs
  147. def load_stimulus_confounds(events: pd.DataFrame, n_scans: int, tr: float) -> pd.DataFrame:
  148. """Generate extra confounds based on stimulus properties."""
  149. coco_df = get_coco_df()
  151. text_cols = ["sentence_length", "amr_n_nodes"]
  152. img_cols = ["coco_a_nodes", "ic_score", "aspect_ratio", "coco_person"]
  153. all_cols = text_cols + img_cols
  155. confounds = pd.DataFrame(0.0, index=range(n_scans), columns=all_cols)
  156. txt_df, img_df = get_stimulus_features_lookup(coco_df)
  158. for _, row in events.iterrows():
  159. data = get_stimulus_data(
  160. modality=row.get("modality"),
  161. stimulus=row.get("stimulus"),
  162. cocoid=row.get("cocoid"),
  163. txt_df=txt_df,
  164. img_df=img_df,
  165. )
  167. if data is None:
  168. continue
  170. start_tr = int(round(row["onset"] / tr))
  171. n_trs = int(round(row["duration"] / tr))
  172. end_tr = min(start_tr + n_trs, n_scans)
  174. if start_tr >= n_scans:
  175. continue
  177. cols = text_cols if row.get("modality") == "text" else img_cols
  178. valid_cols = [c for c in cols if c in data]
  179. if valid_cols:
  180. confounds.iloc[
  181. start_tr:end_tr, confounds.columns.get_indexer(valid_cols) # type: ignore
  182. ] = data[valid_cols].values
  184. return confounds
  187. def conjunction_map(map_a: Path, map_b: Path) -> Path:
  188. """Compute a simple conjunction (logical AND) of two thresholded maps."""
  190. img_a = image.math_img("img != 0", img=map_a)
  191. img_b = image.math_img("img != 0", img=map_b)
  192. conj = image.math_img("img1 * img2", img1=img_a, img2=img_b)
  193. out_file = map_a.with_name("conjunction_img_txt.nii.gz")
  194. save_brain_map(conj, out_file)
  195. return out_file
  198. def split_runs_by_session(runs: Iterable[Tuple[Path, Path, Path]]) -> Dict[str, List[Tuple[Path, Path, Path]]]:
  199. """Group runs by session token for downstream localizer logic."""
  201. grouped: Dict[str, List[Tuple[Path, Path, Path]]] = {}
  202. for bold, events, confounds in runs:
  203. run_id = derive_run_id(bold)
  204. session = run_id.split("_")[1]
  205. grouped.setdefault(session, []).append((bold, events, confounds))
  206. return grouped
  210. def get_aspect_ratio(filepath: str):
  211. """Get the aspect ratio of the image from the local directory.
  213. :param filepath: The path to the image file
  214. :type filepath: str
  215. :return: The aspect ratio of the image
  216. :rtype: float
  217. """
  218. # Load the image
  219. try:
  220. with Image.open(filepath) as img:
  221. width, height = img.size
  222. return width / height
  223. except Exception as e: # noqa
  224. return 0.0
  227. def get_amr_graph_depth(
  228. amr_graph: str,
  229. return_graph=False,
  230. ) -> Union[int, Tuple[int, nx.DiGraph]]:
  231. """Get the depth of the AMR graph for a given example.
  233. :param amr_graph: The AMR graph to get the depth for (output of a spacy doc._.to_amr()[0] call)
  234. :type amr_graph: str
  235. :param return_graph: Whether to return the networkx graph, defaults to False
  236. :type return_graph: bool, optional
  237. :return: The maximum "depth" of the AMR graph (longest shortest path)
  238. :rtype: int
  239. """
  240. # Convert to a Penman graph (with de-inverted edges)
  241. penman_graph = penman.decode(amr_graph)
  242. # Convert to a nx graph, first initialize the nx graph
  243. nx_graph = nx.DiGraph()
  244. # Add edges
  245. for e in penman_graph.edges():
  246. nx_graph.add_edge(e.source, e.target)
  247. # Get the characteristic path length of the graph
  248. amr_graph_depth = (
  249. max([max(nx.shortest_path_length(nx_graph, source=n).values()) for n in nx_graph.nodes()])
  250. if nx.number_of_nodes(nx_graph) > 0
  251. else 0
  252. )
  254. if return_graph:
  255. return amr_graph_depth, nx_graph
  256. else:
  257. return amr_graph_depth
  260. def walk_tree(
  261. node: Token,
  262. depth: int,
  263. ) -> int:
  264. """Walk the dependency parse tree and return the maximum depth.
  266. :param node: The current node in the tree
  267. :type node: spacy.tokens.Token
  268. :param depth: The current depth in the tree
  269. :type depth: int
  270. :return: The maximum depth in the tree
  271. :rtype: int
  272. """
  273. if node.n_lefts + node.n_rights > 0:
  274. return max(walk_tree(child, depth + 1) for child in node.children)
  275. else:
  276. return depth
  279. def derive_text_depth_features(
  280. examples: Dict[str, List],
  281. nlp: Language,
  282. ) -> Dict[str, List]:
  283. """Get the depth of the dep parse tree, number of verbs and "depth" of the AMR graph of an example caption.
  285. The AMR model needs to be downloaded separately, see https://github.com/bjascob/amrlib-models.
  287. :param examples: A batch of hf dataset examples
  288. :type examples: Dict[str, List]
  289. :param nlp: Spacy pipeline to use, initialized using nlp = spacy.load("en_core_web_trf")
  290. :type nlp: spacy.language.Language
  291. :return: The batch with the added features
  292. :rtype: Dict[str, List]
  293. """
  294. result: Dict = {
  295. "parse_tree_depth": [],
  296. "n_verbs": [],
  297. "amr_graph_depth": [],
  298. "amr_graph": [],
  299. "amr_n_nodes": [],
  300. "amr_n_edges": [],
  301. }
  302. doc_batched = nlp.pipe(examples["sentences_raw"])
  304. for doc in doc_batched:
  305. # Also derive the AMR graph for the caption and derive its depth
  306. amr_graph = doc._.to_amr()[0] # type: ignore
  307. try:
  308. amr_depth, amr_graph_obj = get_amr_graph_depth(amr_graph, return_graph=True) # type: ignore
  309. n_nodes = nx.number_of_nodes(amr_graph_obj)
  310. n_edges = nx.number_of_edges(amr_graph_obj)
  311. amr_graph_arr = nx.to_numpy_array(amr_graph_obj)
  312. except DecodeError:
  313. amr_depth = 0
  314. n_nodes = 0
  315. n_edges = 0
  316. amr_graph_arr = nx.to_numpy_array(nx.DiGraph())
  317. # Determine the depth of the dependency parse tree
  318. result["parse_tree_depth"].append(walk_tree(next(doc.sents).root, 0))
  319. result["n_verbs"].append(len([token for token in doc if token.pos_ == "VERB"]))
  320. result["amr_graph_depth"].append(amr_depth)
  321. result["amr_graph"].append(amr_graph_arr)
  322. result["amr_n_nodes"].append(n_nodes)
  323. result["amr_n_edges"].append(n_edges)
  325. return examples | result
  328. def dependency_parse_to_nx(
  329. sents: List[Doc],
  330. ):
  331. """Convert spaCy sentence objects into a NetworkX directed graph representing the dependency parse tree.
  333. :param sents: A list of spaCy sentence objects
  334. :type sents: List[spacy.tokens.Doc]
  335. :return: A NetworkX directed graph representing the dependency parse tree
  336. :rtype: nx.DiGraph
  337. """
  338. # Create a directed graph
  339. graph = nx.DiGraph()
  341. # Iterate over sentences
  342. for sent in sents:
  343. for token in sent:
  344. # Add node for the token with attributes
  345. graph.add_node(token.i, text=token.text, pos=token.pos_, tag=token.tag_)
  346. # Add edge from head to child (if not the root token)
  347. if token.head != token: # Avoid self-loop for the root
  348. graph.add_edge(token.head.i, token.i, dep=token.dep_)
  350. return graph
  353. def flatten_examples(
  354. examples: Dict[str, List],
  355. flatten_col_names: List[str] = ["sentences_raw", "sentids"],
  356. ) -> Dict[str, List]:
  357. """Flattens the examples in the dataset.
  359. :param examples: The examples to flatten
  360. :type examples: Dict[str, List]
  361. :param flatten_col_names: The column names to flatten, defaults to ["sentences_raw", "sentids"]
  362. :type flatten_col_names: List[str]
  363. :return: The flattened examples
  364. :rtype: Dict[str, List]
  365. """
  366. flattened_data = {}
  367. number_of_sentences = [len(sents) for sents in examples[flatten_col_names[0]]]
  368. for key, value in examples.items():
  369. if key in flatten_col_names:
  370. flattened_data[key] = [sent for sent_list in value for sent in sent_list]
  371. else:
  372. flattened_data[key] = np.repeat(value, number_of_sentences).tolist()
  373. return flattened_data
  376. def apply_gamma_correction(
  377. image: Image.Image,
  378. target_mean=128.0,
  379. ) -> Image.Image:
  380. """Apply gamma correction to an image.
  382. :param image: The image to apply gamma correction to
  383. :type image: PIL.Image.Image
  384. :param target_mean: The target mean brightness of the image, defaults to 128.0
  385. :type target_mean: float, optional
  386. :return: The image with gamma correction applied
  387. :rtype: PIL.Image.Image
  388. """
  389. # Convert the PIL image to a numpy array
  390. img_array = np.array(image)
  392. # Calculate the current mean brightness of the image
  393. current_mean = np.mean(img_array)
  395. # Calculate the gamma value to adjust the mean to the target mean
  396. # Avoid division by zero
  397. if current_mean > 0:
  398. gamma = np.log(target_mean) / np.log(current_mean)
  399. # Apply gamma correction to the image
  400. corrected_image = ImageOps.autocontrast(image, cutoff=gamma) # type: ignore
  401. return corrected_image
  402. else:
  403. return image
  406. def save_brain_map(
  407. img: Union[nib.nifti1.Nifti1Image, object],
  408. output_path: Union[str, Path],
  409. is_z_map: bool = False,
  410. voxel_p: float = DEFAULT_VOXEL_P,
  411. cluster_threshold: int = DEFAULT_CLUSTER_THRESHOLD,
  412. make_surface_plot: bool = True,
  413. surface_plot_kwargs: Optional[Dict] = None,
  414. sided: str = "positive",
  415. ) -> Optional[nib.nifti1.Nifti1Image]:
  416. """Save a brain map to disk, generate mosaic plots, and cluster tables.
  418. For z-score/stat maps (``is_z_map=True``), apply one-sided FPR thresholding
  419. with ``threshold_stats_img`` (using ``voxel_p`` and ``cluster_threshold``).
  420. Set ``sided="positive"`` to keep only positive effects, ``sided="negative"``
  421. for negative effects, or ``sided="both"`` to save both directions (files
  422. suffixed with ``_pos_thr`` and ``_neg_thr``).
  424. :param img: The brain map image
  425. :param output_path: Where to save the image
  426. :param is_z_map: Whether the image is a z-/stat-map that should be thresholded
  427. :param voxel_p: Two-sided voxel-wise p-value used for thresholding
  428. :param make_surface_plot: Whether to save a surface plot
  429. :param surface_plot_kwargs: Extra args for ``generate_surface_plots``
  430. :param sided: Direction for one-sided thresholding (positive, negative, both)
  431. :returns: Thresholded positive image if available, otherwise None
  432. """
  433. output_path = Path(output_path)
  434. output_path.parent.mkdir(parents=True, exist_ok=True)
  436. def _base_path(path: Path) -> str:
  437. str_path = str(path)
  438. if str_path.endswith(".nii.gz"):
  439. return str_path[:-7]
  440. if str_path.endswith(".nii"):
  441. return str_path[:-4]
  442. return str(path.with_suffix(""))
  444. def _plot(target_img: object, target_path: Path, cmap: Optional[str] = None) -> None:
  445. base = _base_path(target_path)
  446. try:
  447. display = plotting.plot_stat_map(
  448. target_img,
  449. display_mode="mosaic",
  450. cut_coords=5,
  451. cmap=cmap, # type: ignore
  452. )
  453. display.savefig(f"{base}_mosaic.png") # type: ignore
  454. display.close() # type: ignore
  455. except Exception as e:
  456. logger.error(f"Failed to plot brain map: {e}")
  458. # Save the Nifti image
  459. if hasattr(img, "to_filename"):
  460. img.to_filename(output_path) # type: ignore
  461. else:
  462. nib.save(img, output_path) # type: ignore
  464. _plot(img, output_path)
  466. thresholded_img = None
  467. if is_z_map:
  468. try:
  469. do_pos = sided in ("positive", "both")
  470. do_neg = sided in ("negative", "both")
  472. def _save_thr(target_img: object, suffix: str, flip_back: bool = False, cmap: Optional[str] = None) -> object:
  473. thr_img, _ = threshold_stats_img(
  474. target_img,
  475. alpha=voxel_p,
  476. height_control="fpr",
  477. cluster_threshold=cluster_threshold,
  478. two_sided=False,
  479. )
  480. if flip_back:
  481. thr_img = image.math_img("-img", img=thr_img)
  482. if str(output_path).endswith(".nii.gz"):
  483. thr_path = output_path.with_name(output_path.name.replace(".nii.gz", f"_{suffix}.nii.gz"))
  484. else:
  485. thr_path = output_path.with_name(f"{output_path.stem}_{suffix}{output_path.suffix}")
  486. thr_img.to_filename(thr_path) # type: ignore
  487. _plot(thr_img, thr_path, cmap=cmap)
  488. return thr_img
  490. if do_pos:
  491. thresholded_img = _save_thr(img, "pos_thr", cmap="Reds")
  492. if do_neg:
  493. _save_thr(image.math_img("-img", img=img), "neg_thr", flip_back=True, cmap="Blues_r")
  494. except Exception as e:
  495. logger.error(f"Failed to threshold brain map: {e}")
  497. # Save surface plot
  498. if make_surface_plot:
  499. from compositionality_study.visualization.visualize_brain_maps import generate_surface_plots
  500. surface_plot_kwargs = surface_plot_kwargs or {}
  501. try:
  502. generate_surface_plots(
  503. nii_file=output_path,
  504. output_dir=output_path.parent,
  505. **surface_plot_kwargs
  506. )
  507. except Exception as e:
  508. logger.error(f"Failed to generate surface plot: {e}")
  510. return thresholded_img # type: ignore

utils.py at commit bb03182, under MIT · at the source

Overview

Authors: Helena Balabin1,2, Antonietta Gabriella Liuzzi1, Kevin Statz1, Marie-Francine Moens2, Patrick Dupont1, Rik Vandenberghe1
ORCID iDs: Helena Balabin, Antonietta Gabriella Liuzzi, Kevin Statz, Marie-Francine Moens, Patrick Dupont, Rik Vandenberghe
  1. Laboratory for Cognitive Neurology, Department of Neurosciences, Leuven Brain Institute, KU Leuven, Leuven, Belgium
  2. Language Intelligence and Information Retrieval Lab, Department of Computer Science, KU Leuven, Leuven, Belgium
Institutions: KU Leuven (Belgium)
Journal: Neurobiology of language (Cambridge, Mass.), volume 7, article NOL.a.271
Dates: received 27 February 2025; accepted 28 April 2026; published online 9 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1162/nol.a.271 · PMID 42644209 · PMCID PMC13506228 · OpenAlex W7160494749
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: fMRI (modality)
Methods: Statistics, Machine learning, Connectivity, fMRI & imaging, Single-unit activity, calcium imaging, Smoothing, state filtering, decompositions
Keywords: compositionality, functional magnetic resonance imaging, multivariate pattern analysis
Topic: Neurobiology of Language and Bilingualism (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: Fonds Wetenschappelijk Onderzoek (10.13039/501100003130, 1154623N, 501100003130, 1247821N); KU Leuven (501100004040, C14/21/109)
Citations: not cited yet (Europe PMC); 78 references in the paper

Abstract

Compositionality enables us to derive the meaning of a complex whole from the syntax and semantics of its individual parts. While compositional processing has primarily been studied in language, similar principles apply to visual scenes, in which meaning can be derived from their individual parts and the relationships between them. In this study, we therefore aimed to explore whether there is a shared neural basis for compositional processing in text and images. Based on abstract meaning representation graphs for text, commonly used to capture “who is doing what to whom”, and action graph annotations for visual scenes, we defined an analogous graph depth-based notion of compositional complexity. By conducting a functional magnetic resonance imaging experiment in which participants view text-image pairs from the Common Objects in Context-Actions data set, we aimed to identify brain activity patterns related to compositional processing through a combination of univariate and multivariate pattern analysis. Specifically, we hypothesized that there are shared brain regions—such as the pars opercularis, pars triangularis or anterior temporal lobe—involved in processing compositional complexity across text and images. Alternatively, there might exist distinct regions for processing compositional complexity in text and images, without any substantial cross-modal overlap. Our results provided no evidence in support of either hypothesis: no significant neural responses to compositional complexity were observed, in either a shared or modality-specific manner. Given the rigorous control of confounds in our study design, the operationalization of compositional complexity itself may underlie the null result.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 12 matches between paragraphs and lines of code.

lcn-kul/compositionality-study

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: bb03182371decdfede04bfac39e5dc335e2a789d, 22 April 2026
Languages: Python (31)
Size: 52 files, 31 scripts
Software Heritage: archived
Found in: “DATA AND CODE AVAILABILITY STATEMENTS”
Holds: README, license file, CITATION.cff, environment (poetry.lock, pyproject.toml, setup.cfg, setup.py, uv.lock), continuous integration, documentation
Not found: tests
Tools: pandas (15 files), NumPy (12 files), Pillow (6 files), Matplotlib (5 files), Nilearn (5 files), NetworkX (4 files), NiBabel (4 files), SciPy (4 files), seaborn (3 files), GLMsingle (2 files), OpenCV (2 files), PyTorch (2 files), Pingouin (1 file), PsychoPy (1 file), scikit-learn (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
33 files
At the source: github.com/lcn-kul/compositionality-study

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 31 scripts, each with its path and the digest of its content;
  • 12 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data and code availability statements

The approved Stage 1 protocol, laboratory log, stimuli and behavioral results can be found on the Open Science Framework: https://osf.io/ta45d/overview?view_only=439003edb48c4053ab464d000894fa3e. Furthermore, the pre-processed study data supporting the reported analysis is available at Zenodo: https://zenodo.org/records/18831683 and https://zenodo.org/records/18832298. All code used for the pre-registered analyses is publicly available as well: https://github.com/lcn-kul/compositionality-study.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

  • Funding: added Fonds Wetenschappelijk Onderzoek: 10.13039/501100003130, 1154623N, 501100003130, 1247821N; KU Leuven: 501100004040, C14/21/109

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 6 authors, 3 keywords, 71 references.

Cite

This paper

Balabin, H., Liuzzi, A. G., Statz, K., Moens, M.-F., Dupont, P., & Vandenberghe, R. (2026). Compositional Complexity in Text and Images. Neurobiology of language (Cambridge, Mass.), 7, NOL.a.271. https://doi.org/10.1162/nol.a.271

BibTeX

@article{balabin2026compositional,
author = {Balabin, Helena and Liuzzi, Antonietta Gabriella and Statz, Kevin and Moens, Marie-Francine and Dupont, Patrick and Vandenberghe, Rik},
title = {{Compositional Complexity in Text and Images}},
journal = {Neurobiology of language (Cambridge, Mass.)},
year = {2026},
month = jul,
volume = {7},
pages = {NOL.a.271},
publisher = {MIT Press},
issn = {2641-4368},
doi = {10.1162/nol.a.271},
url = {https://doi.org/10.1162/nol.a.271},
pmid = {42644209},
pmcid = {PMC13506228}
}

RIS

TY - JOUR
AU - Balabin, Helena
AU - Liuzzi, Antonietta Gabriella
AU - Statz, Kevin
AU - Moens, Marie-Francine
AU - Dupont, Patrick
AU - Vandenberghe, Rik
TI - Compositional Complexity in Text and Images
T2 - Neurobiology of language (Cambridge, Mass.)
J2 - Neurobiol Lang (Camb)
PY - 2026
DA - 2026/07/09
VL - 7
SP - NOL.a.271
SN - 2641-4368
PB - MIT Press
DO - 10.1162/nol.a.271
UR - https://doi.org/10.1162/nol.a.271
LA - en
ER -

CSL-JSON

{
"id": "10.1162/nol.a.271",
"type": "article-journal",
"title": "Compositional Complexity in Text and Images",
"container-title": "Neurobiology of language (Cambridge, Mass.)",
"author": [
{
"family": "Balabin",
"given": "Helena"
},
{
"family": "Liuzzi",
"given": "Antonietta Gabriella"
},
{
"family": "Statz",
"given": "Kevin"
},
{
"family": "Moens",
"given": "Marie-Francine"
},
{
"family": "Dupont",
"given": "Patrick"
},
{
"family": "Vandenberghe",
"given": "Rik"
}
],
"container-title-short": "Neurobiol Lang (Camb)",
"volume": "7",
"page": "NOL.a.271",
"DOI": "10.1162/nol.a.271",
"PMID": "42644209",
"PMCID": "PMC13506228",
"ISSN": "2641-4368",
"publisher": "MIT Press",
"URL": "https://doi.org/10.1162/nol.a.271",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
9
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.7554/elife.107933 [code]
Modality-agnostic decoding of vision and language from fMRI.
Journal: eLife
In common: Nilearn, OpenCV, Pillow, 8 other tools, fMRI, 7 references
[2] doi:10.1038/s41597-026-07248-6 [code]
A large-scale fMRI dataset for vision-language semantic association.
Journal: Scientific data
In common: GLMsingle, Nilearn, OpenCV, 8 other tools, fMRI, 4 references
[3] doi:10.1523/jneurosci.0038-26.2026 [code]
Multidimensional Feature Tuning in Category Selective Areas of Human Visual Cortex.
Journal: The Journal of neuroscience : the official journal of the Society for Neuroscience
In common: Pillow, NiBabel, PyTorch, 6 other tools, fMRI, 6 references
[4] doi:10.1162/imag.a.1309 [code]
Probing the content of semantic representations in body-selective regions.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: OpenCV, Pillow, NiBabel, 7 other tools, 5 references
[5] doi:10.1038/s41467-026-76098-y [code]
A single computational objective can produce specialization of streams in visual cortex.
Journal: Nature communications
In common: Pillow, NiBabel, PyTorch, 6 other tools, 5 references
[6] doi:10.1038/s42003-026-10169-0 [code]
Shared representations in brains and models reveal a two-route cortical organization during scene perception.
Journal: Communications biology
In common: Pillow, NiBabel, PyTorch, 6 other tools, 5 references
[7] doi:10.1162/imag.a.1286 [code]
Behavioral imitation with artificial neural networks leads to personalized models of brain dynamics during videogame play.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: Nilearn, Pillow, NiBabel, 7 other tools, fMRI, 4 references
[8] doi:10.1162/imag.a.1256 [code]
Gamer in the scanner: Event-related analysis of fMRI activity during retro videogame play guided by automated annotations of game content.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: Nilearn, Pillow, NiBabel, 7 other tools, fMRI, 4 references
[9] doi:10.1167/jov.26.5.7 [code]
Representations in vision and language converge in a shared, multidimensional space of perceived similarities.
Journal: Journal of vision
In common: Nilearn, OpenCV, Pillow, 8 other tools, 2 references
[10] doi:10.1007/s12021-026-09803-3 [code]
NeuroFusion: A Unified Framework for Generalized Visual Stimulus Decoding from fMRI Across Datasets and Subjects.
Journal: Neuroinformatics
In common: Nilearn, Pillow, NiBabel, 6 other tools, fMRI, 3 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.