Talking to the Brain: Using Large Language Models as Proxies to Model Brain Semantic Features.

Code ↔ Paper

The paper beside its authors' code: matches between them have not been computed for this paper yet.

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The paper is loaded by your browser, with JavaScript: read it at doi.org or on Europe PMC.

The authors' code

Python · 15 lines · 503 B · MIT

import os
here = os.path.abspath(os.path.dirname(__file__))
about = {}
with open(os.path.join(here, "pyecharts", "_version.py")) as f:
exec(f.read(), about)
UNINSTALL = "{} uninstall pyecharts -y"
INSTALL = "{} install -U dist/pyecharts-{}-py3-none-any.whl --no-cache-dir"
os.system("python setup.py bdist_wheel")
os.system(UNINSTALL.format("pip"))
os.system(UNINSTALL.format("pip3"))
os.system(INSTALL.format("pip", about["__version__"]))
os.system(INSTALL.format("pip3", about["__version__"]))

install.py at commit c50da2a, under MIT · at the source

Overview

Authors: Xin Liu^1,2,3,4,5, Ziyue Zhang^1,2,3,4,5, Jingxin Nie^1,2,3,4,5

ORCID iDs: Ziyue Zhang, Jingxin Nie

Philosophy and Social Science Laboratory of Reading and Development in Children and Adolescents (South China Normal University), Ministry of Education Center for Studies of Psychological Application, South China Normal University Guangzhou China
Center for Studies of Psychological Application South China Normal University Guangzhou China
Key Laboratory of Brain, Cognition and Education Sciences (South China Normal University), Ministry of Education Guangzhou China
School of Psychology South China Normal University Guangzhou China
Guangdong Key Laboratory of Mental Health and Cognitive Science South China Normal University Guangzhou China

Institutions: South China Normal University (China)

Journal: Human brain mapping, volume 47, issue 10, article e70588

Dates: received 12 September 2025; accepted 15 June 2026; published online 6 July 2026; in print July 2026

Type: Other · Language: English

License: CC BY

Identifiers: DOI 10.1002/hbm.70588 · PMID 42403029 · PMCID PMC13334217 · OpenAlex W7167517032

Open access: gold, a free copy (OpenAlex)

Status: code verified

Categories: human (organism), methods / tools (subfield)

Methods: Connectivity, Machine learning, Statistics, fMRI & imaging

MeSH: Brain*, Brain Mapping*, Models, Neurological*, Semantics*, Adult, Female, Humans, Large Language Models, Magnetic Resonance Imaging, Male, Young Adult (* major topic)

Journal subjects: Technical Report

Topic: Face Recognition and Perception (Cognitive Neuroscience, Neuroscience), according to OpenAlex

Funding: Research Center for Brain Cognition and Human Development (2024B0303390003); Key-Area Research and Development Program of Guangdong Province (2019B030335001); Striving for the First-Class, Improving Weak Links and Highlighting Features (SIH) Key Discipline for Psychology in South China Normal University

Citations: cited by 1 paper (Europe PMC); 43 references in the paper

Abstract

Utilizing naturalistic stimuli while maintaining experimental rigor presents a significant challenge in neuroimaging research, primarily due to the complexities of annotation and high‐dimensional analysis. To address this, we introduce a novel paradigm that leverages multimodal large language models (LLMs) as high‐throughput, automated annotation tools to extract human‐understandable semantic features. This “talking to the brain” approach employs a Visual Question Answering (VQA) strategy to transform complex visual scenes into structured semantic feature vectors, which are then used to predict voxel‐wise brain activity. As a critical methodological validation, our model successfully replicated established neural correlates for categories such as face and building. Expanding beyond these benchmarks, we mapped the cortical activation patterns of 80 diverse semantic labels and constructed a data‐driven brain semantic similarity space. This space revealed robust clusters reflecting both functional categories and real‐world contextual associations. This innovative methodology offers a scalable solution for investigating brain semantic organization, overcoming the limitations of manual annotation and paving the way for more ecologically valid explorations of human cognition.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above.

pyecharts/pyecharts

License: MIT

State: the link answers, verified on 27 September 2026

Evidence: files inventoried

Commit: c50da2aa877bb4ce576b15b5be3b2ad0c8aa33c6, 10 February 2026

Languages: Python (135)

Size: 184 files, 135 scripts

Software Heritage: archived

Found in: the text, “Semantic Similarity Space Construction”

Holds: README, license file, environment (pyproject.toml, requirements-dev.txt, requirements.txt, setup.py, uv.lock, test/requirements.txt), tests, continuous integration

Not found: CITATION.cff, documentation

Tools: NumPy (1 file), pandas (1 file), Pillow (1 file)

Availability: 1 check, the latest on 27 September 2026: the link answers

27 September 2026: the link answers

137 files

install.py, Python, 15 lines
pyecharts/__init__.py, Python, 2 lines
pyecharts/_version.py, Python, 2 lines
pyecharts/charts/__init__.py, Python, 51 lines
pyecharts/charts/base.py, Python, 146 lines
pyecharts/charts/basic_charts/__init__.py, Python, 1 line
pyecharts/charts/basic_charts/amap.py, Python, 91 lines
pyecharts/charts/basic_charts/bar.py, Python, 123 lines
pyecharts/charts/basic_charts/bmap.py, Python, 92 lines
pyecharts/charts/basic_charts/boxplot.py, Python, 116 lines
pyecharts/charts/basic_charts/calendar.py, Python, 62 lines
pyecharts/charts/basic_charts/chord.py, Python, 74 lines
pyecharts/charts/basic_charts/custom.py, Python, 139 lines
pyecharts/charts/basic_charts/effectscatter.py, Python, 104 lines
pyecharts/charts/basic_charts/funnel.py, Python, 115 lines
pyecharts/charts/basic_charts/gauge.py, Python, 88 lines
pyecharts/charts/basic_charts/geo.py, Python, 356 lines
pyecharts/charts/basic_charts/gmap.py, Python, 77 lines
pyecharts/charts/basic_charts/graph.py, Python, 141 lines
pyecharts/charts/basic_charts/heatmap.py, Python, 98 lines
pyecharts/charts/basic_charts/kline.py, Python, 94 lines
pyecharts/charts/basic_charts/line.py, Python, 127 lines
pyecharts/charts/basic_charts/liquid.py, Python, 65 lines
pyecharts/charts/basic_charts/lmap.py, Python, 66 lines
pyecharts/charts/basic_charts/map.py, Python, 126 lines
pyecharts/charts/basic_charts/parallel.py, Python, 87 lines
pyecharts/charts/basic_charts/pictorialbar.py, Python, 81 lines
pyecharts/charts/basic_charts/pie.py, Python, 124 lines
pyecharts/charts/basic_charts/polar.py, Python, 143 lines
pyecharts/charts/basic_charts/radar.py, Python, 123 lines
pyecharts/charts/basic_charts/sankey.py, Python, 92 lines
pyecharts/charts/basic_charts/scatter.py, Python, 115 lines
pyecharts/charts/basic_charts/sunburst.py, Python, 79 lines
pyecharts/charts/basic_charts/themeriver.py, Python, 70 lines
pyecharts/charts/basic_charts/tree.py, Python, 116 lines
pyecharts/charts/basic_charts/treemap.py, Python, 97 lines
pyecharts/charts/basic_charts/wordcloud.py, Python, 140 lines
pyecharts/charts/chart.py, Python, 402 lines
pyecharts/charts/composite_charts/__init__.py, Python, 1 line
pyecharts/charts/composite_charts/grid.py, Python, 130 lines
pyecharts/charts/composite_charts/page.py, Python, 221 lines
pyecharts/charts/composite_charts/tab.py, Python, 164 lines
pyecharts/charts/composite_charts/timeline.py, Python, 156 lines
pyecharts/charts/mixins.py, Python, 25 lines
pyecharts/charts/three_axis_charts/__init__.py, Python, 1 line
pyecharts/charts/three_axis_charts/bar3D.py, Python, 18 lines
pyecharts/charts/three_axis_charts/graph_gl.py, Python, 51 lines
pyecharts/charts/three_axis_charts/line3D.py, Python, 18 lines
pyecharts/charts/three_axis_charts/lines3D.py, Python, 66 lines
pyecharts/charts/three_axis_charts/map3D.py, Python, 242 lines
pyecharts/charts/three_axis_charts/map_globe.py, Python, 54 lines
pyecharts/charts/three_axis_charts/scatter3D.py, Python, 18 lines
pyecharts/charts/three_axis_charts/surface3D.py, Python, 19 lines
pyecharts/commons/__init__.py, Python, 1 line
pyecharts/commons/utils.py, Python, 115 lines
pyecharts/components/__init__.py, Python, 2 lines
pyecharts/components/image.py, Python, 56 lines
pyecharts/components/table.py, Python, 63 lines
pyecharts/datasets/__init__.py, Python, 138 lines
pyecharts/exceptions.py, Python, 15 lines
pyecharts/faker.py, Python, 102 lines
pyecharts/globals.py, Python, 183 lines
pyecharts/options/__init__.py, Python, 156 lines
pyecharts/options/charts_options.py, Python, 1,775 lines
pyecharts/options/global_options.py, Python, 2,098 lines
pyecharts/options/series_options.py, Python, 561 lines
pyecharts/render/__init__.py, Python, 1 line
pyecharts/render/display.py, Python, 86 lines
pyecharts/render/engine.py, Python, 141 lines
pyecharts/render/snapshot.py, Python, 99 lines
pyecharts/scaffold/__init__.py, Python, 1 line
pyecharts/types.py, Python, 129 lines
setup.py, Python, 105 lines
test.py, Python, 13 lines
test/__init__.py, Python, 20 lines
test/test_amap.py, Python, 89 lines
test/test_bar.py, Python, 359 lines
test/test_bar3d.py, Python, 83 lines
test/test_base.py, Python, 125 lines
test/test_bmap.py, Python, 138 lines
test/test_boxplot.py, Python, 94 lines
test/test_calendar.py, Python, 134 lines
test/test_chart.py, Python, 250 lines
test/test_chart_options.py, Python, 181 lines
test/test_chord.py, Python, 48 lines
test/test_custom.py, Python, 74 lines
test/test_datasets.py, Python, 90 lines
test/test_display.py, Python, 53 lines
test/test_effectscatter.py, Python, 30 lines
test/test_engine.py, Python, 80 lines
test/test_exception.py, Python, 100 lines
test/test_faker.py, Python, 21 lines
test/test_funnel.py, Python, 33 lines
test/test_gauge.py, Python, 38 lines
test/test_geo.py, Python, 423 lines
test/test_global_options.py, Python, 503 lines
test/test_globals.py, Python, 20 lines
test/test_gmap.py, Python, 85 lines
test/test_graph.py, Python, 137 lines
test/test_graph_gl.py, Python, 63 lines
test/test_graphic.py, Python, 60 lines
test/test_grid.py, Python, 647 lines
test/test_heatmap.py, Python, 23 lines
test/test_image.py, Python, 43 lines
test/test_kline.py, Python, 79 lines
test/test_line.py, Python, 145 lines
test/test_line3d.py, Python, 38 lines
test/test_lines3d.py, Python, 110 lines
test/test_liquid.py, Python, 46 lines
test/test_lmap.py, Python, 64 lines
test/test_map.py, Python, 233 lines
test/test_map3d.py, Python, 330 lines
test/test_map_globe.py, Python, 48 lines
test/test_mixins.py, Python, 24 lines
test/test_page.py, Python, 183 lines
test/test_parallel.py, Python, 119 lines
test/test_pictorialbar.py, Python, 26 lines
test/test_pie.py, Python, 90 lines
test/test_polar.py, Python, 57 lines
test/test_radar.py, Python, 93 lines
test/test_sankey.py, Python, 71 lines
test/test_scatter.py, Python, 235 lines
test/test_scatter3d.py, Python, 27 lines
test/test_series_options.py, Python, 205 lines
test/test_snapshot.py, Python, 118 lines
test/test_sunburst.py, Python, 61 lines
test/test_surface3d.py, Python, 42 lines
test/test_tab.py, Python, 98 lines
test/test_table.py, Python, 51 lines
test/test_themeriver.py, Python, 27 lines
test/test_timeline.py, Python, 193 lines
test/test_tree.py, Python, 60 lines
test/test_treemap.py, Python, 60 lines
test/test_utils.py, Python, 83 lines
test/test_wordcloud.py, Python, 88 lines
LICENSE, License, 21 lines
README.md, Text, 256 lines

At the source: github.com/pyecharts/pyecharts

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
135 scripts, each with its path and the digest of its content;
no match between paragraphs and code yet;
neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

The NSD dataset (Allen et al. 2022) is publicly available on https://naturalscenesdataset.org/. Purdue Movie dataset (Wen et al. 2018) is publicly available on https://purr.purdue.edu/publications/2809/1. All analyses were performed using Python and DPABISurf (Yan et al. 2021). The fMRI data were analyzed with nilearn (https://nilearn.github.io/stable/index.html). Part of model fitting and statistical analysis was conducted using Statsmodels (Seabold and Perktold 2010).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 3 authors, 11 MeSH terms, 3 funders, 25 references.

Cite

This paper

Liu, X., Zhang, Z., & Nie, J. (2026). Talking to the Brain: Using Large Language Models as Proxies to Model Brain Semantic Features. Human brain mapping, 47(10), e70588. https://doi.org/10.1002/hbm.70588

BibTeX

@article{liu2026talking, author = {Liu, Xin and Zhang, Ziyue and Nie, Jingxin}, title = {{Talking to the Brain: Using Large Language Models as Proxies to Model Brain Semantic Features}}, journal = {Human brain mapping}, year = {2026}, month = jul, volume = {47}, number = {10}, pages = {e70588}, publisher = {Wiley}, issn = {1065-9471}, doi = {10.1002/hbm.70588}, url = {https://doi.org/10.1002/hbm.70588}, pmid = {42403029}, pmcid = {PMC13334217} }

RIS

TY - JOUR AU - Liu, Xin AU - Zhang, Ziyue AU - Nie, Jingxin TI - Talking to the Brain: Using Large Language Models as Proxies to Model Brain Semantic Features T2 - Human brain mapping J2 - Hum Brain Mapp PY - 2026 DA - 2026/07/01 VL - 47 IS - 10 SP - e70588 SN - 1065-9471 PB - Wiley DO - 10.1002/hbm.70588 UR - https://doi.org/10.1002/hbm.70588 LA - en ER -

CSL-JSON

{ "id": "10.1002/hbm.70588", "type": "article-journal", "title": "Talking to the Brain: Using Large Language Models as Proxies to Model Brain Semantic Features", "container-title": "Human brain mapping", "author": [ { "family": "Liu", "given": "Xin" }, { "family": "Zhang", "given": "Ziyue" }, { "family": "Nie", "given": "Jingxin" } ], "container-title-short": "Hum Brain Mapp", "volume": "47", "issue": "10", "page": "e70588", "DOI": "10.1002/hbm.70588", "PMID": "42403029", "PMCID": "PMC13334217", "ISSN": "1065-9471", "publisher": "Wiley", "URL": "https://doi.org/10.1002/hbm.70588", "language": "en", "issued": { "date-parts": [ [ 2026, 7, 1 ] ] } }

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.