Larger language models better align with neural representations of natural language.
The 4 matches
- [1] § Materials and methods › Contextual embeddings ↔ scripts/tfsemb_download.py, lines 1 to 45 · score 0.79 · gpt2 xl, GPT Neox, EleutherAI, OPT, models · Report a mismatch
- [2] § Materials and methods › Encoding models ↔ scripts/tfsenc_encoding.py, lines 148 to 195 · score 0.69 · RidgeCV, himalaya, alpha, OLS, fitting, PCA · Report a mismatch
- [3] § Results ↔ scripts/tfsemb_download.py, lines 1 to 45 · score 0.60 · gpt neox, MEDIUM, OPT, XL, model · Report a mismatch
- [4] § Materials and methods › Contextual embeddings ↔ scripts/tfsenc_config.py, lines 39 to 57 · score 0.60 · EleutherAI, GPT Neo, language model · Report a mismatch
Paper
Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
Python · 269 lines · 7.4 KB · no license · 2 matches
- import os
- from transformers import (
- AutoConfig,
- AutoModel,
- AutoModelForCausalLM,
- AutoModelForMaskedLM,
- AutoModelForSeq2SeqLM,
- AutoTokenizer,
- )
- CAUSAL_MODELS = [
- "gpt2",
- "gpt2-large",
- "gpt2-xl",
- "EleutherAI/gpt-neo-125M",
- "EleutherAI/gpt-neo-1.3B",
- "EleutherAI/gpt-neo-2.7B",
- "EleutherAI/gpt-neox-20b",
- "facebook/opt-125m",
- "facebook/opt-350m",
- "facebook/opt-1.3b",
- "facebook/opt-2.7b",
- "facebook/opt-6.7b",
- "facebook/opt-30b",
- "bigscience/bloom",
- ]
- SEQ2SEQ_MODELS = ["facebook/blenderbot_small-90M", "facebook/blenderbot-3B"]
- MLM_MODELS = [
- # "gpt2-xl", # uncomment to run this model with MLM input
- # "gpt2-medium", # uncomment to run this model with MLM input
- "bert-base-uncased",
- "bert-large-uncased",
- "bert-base-cased",
- "bert-large-cased",
- "roberta-base",
- "roberta-large",
- ]
- MODEL_CLASS_MAP = {
- "causal": (CAUSAL_MODELS, AutoModelForCausalLM),
- "seq2seq": (SEQ2SEQ_MODELS, AutoModelForSeq2SeqLM),
- "mlm": (MLM_MODELS, AutoModelForMaskedLM),
- }
- def clean_lm_model_name(item):
- """Remove unnecessary parts from the language model name.
- Args:
- item (str/list): full model name from HF Hub
- Returns:
- (str/list): pretty model name
- Example:
- clean_lm_model_name(EleutherAI/gpt-neo-1.3B) == 'gpt-neo-1.3B'
- """
- if isinstance(item, str):
- return item.split("/")[-1]
- if isinstance(item, list):
- return [clean_lm_model_name(i) for i in item]
- print("Invalid input. Please check.")
- def get_max_context_length(model_name, tokenizer_class=None):
- """Return maximum possible context length for the model/tokenizer
- Args:
- model_name (str): Model name as seen on https://hugginface.co/models.
- tokenizer_class (Tokenizer, optional):
- Tokenizer class to be instantiated for the model.
- Defaults to None.
- Returns:
- int: Maximum allowed context for the model/tokenizer
- """
- tokenizer = download_hf_tokenizer(
- model_name,
- tokenizer_class=tokenizer_class,
- cache_dir=None,
- local_files_only=False,
- )
- return tokenizer.max_len_single_sentence
- def get_model_num_layers(model_name):
- """Return number of hidden layers in the model
- Args:
- model_name (str): Model name as seen on https://hugginface.co/models.
- Returns:
- int: value of {n_layer|num_layers|num_hidden_layers} attribute
- """
- config = AutoConfig.from_pretrained(model_name)
- num_layers = getattr(
- config,
- "n_layer",
- getattr(
- config,
- "num_layers",
- getattr(config, "num_hidden_layers", None),
- ),
- )
- if not num_layers:
- print(f"{model_name} has no hidden layers")
- exit()
- return num_layers
- def download_hf_model(
- model_name, model_class=None, cache_dir=None, local_files_only=False
- ):
- """Download a Huggingface model from the model repository (cache)."""
- if model_class is None:
- model_class = AutoModel
- if cache_dir is None:
- cache_dir = set_cache_dir()
- model = model_class.from_pretrained(
- model_name,
- output_hidden_states=True,
- cache_dir=cache_dir,
- local_files_only=local_files_only,
- )
- return model
- def download_hf_tokenizer(
- model_name, tokenizer_class=None, cache_dir=None, local_files_only=False
- ):
- """Download a Huggingface tokenizer from the model repository (cache)."""
- if tokenizer_class is None:
- tokenizer_class = AutoTokenizer
- if cache_dir is None:
- cache_dir = set_cache_dir()
- tokenizer = tokenizer_class.from_pretrained(
- model_name,
- add_prefix_space=True,
- cache_dir=cache_dir,
- local_files_only=local_files_only,
- )
- return tokenizer
- def download_tokenizer_and_model(
- CACHE_DIR, tokenizer_class, model_class, model_name, local_files_only
- ):
- """Cache (or load) the model and tokenizer from the model repository (or cache).
- Args:
- CACHE_DIR (str): path where the model and tokenizer will be cached.
- tokenizer_class (Tokenizer): Tokenizer class to be instantiated for the model.
- model_class (Huggingface Model): Model class corresponding to model_name.
- model_name (str): Model name as seen on https://hugginface.co/models.
- local_files_only (bool, optional): False (Default) if caching.
- True if loading from cache.
- Returns:
- tuple: (tokenizer, model)
- """
- print("Downloading model")
- model = download_hf_model(model_name, model_class, CACHE_DIR, local_files_only)
- print("Downloading tokenizer")
- tokenizer = download_hf_tokenizer(
- model_name, tokenizer_class, CACHE_DIR, local_files_only
- )
- return (model, tokenizer)
- def set_cache_dir():
- CACHE_DIR = os.path.join(os.path.dirname(os.getcwd()), ".cache")
- os.makedirs(CACHE_DIR, exist_ok=True)
- return CACHE_DIR
- def get_models_and_class(model_name):
- """Return the appropriate model class and model to download
- Args:
- model_name (str): Model name as seen on https://hugginface.co/models.
- Returns:
- models (list): model name or models in the same class
- mod_class (Huggingface Model): Model class corresponding to model_name.
- """
- models, mod_class = None, None
- for model_key, (model_list, model_class) in MODEL_CLASS_MAP.items():
- if model_name == model_key:
- models, mod_class = model_list, model_class
- break
- elif model_name in model_list:
- models, mod_class = [model_name], model_class
- break
- else:
- continue
- if not models or not model_class:
- print("Invalid Model List or Model Class")
- return models, mod_class
- def download_tokenizers_and_models(model_name=None, local_files_only=False, debug=True):
- """This function downloads the tokenizer and model for the specified model name.
- Args:
- model_name (str, optional): Model name as seen on https://hugginface.co/models.
- Defaults to None.
- local_files_only (bool, optional): False (Default) if caching.
- True if loading from cache.
- debug (bool, optional): Check if caching was successful. Defaults to True.
- Returns:
- dict: Dictionary with model name as key and (tokenizer, model) as value.
- """
- CACHE_DIR = set_cache_dir()
- if model_name is None:
- print("Input argument cannot be empty")
- return
- models, model_class = get_models_and_class(model_name)
- model_dict = {}
- for model_name in models:
- print(f"Model Name: {model_name}")
- model_dict[model_name] = download_tokenizer_and_model(
- CACHE_DIR,
- AutoTokenizer,
- model_class,
- model_name,
- local_files_only,
- )
- # check if caching was successful
- if debug:
- print("Checking if model has been cached successfully")
- try:
- download_tokenizer_and_model(
- CACHE_DIR,
- AutoTokenizer,
- model_class,
- model_name,
- True,
- )
- except:
- print(f"Caching of {model_name} failed")
- return model_dict
- if __name__ == "__main__":
- download_tokenizers_and_models("causal", local_files_only=False, debug=True)
tfsemb_download.py at commit b7a6fcb, no license · at the source
Overview
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, United States
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, United States
- New York University Grossman School of Medicine, New York, United States
- Business School, Data Science Department and Cognitive Science Department, Hebrew University, Jerusalem, Israel
Abstract
Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have rapidly grown in complexity, leading to improved language processing capabilities. Here, we utilized several families of transformer-based LLMs to investigate the relationship between model size and their ability to capture linguistic information in the human brain. Crucially, a subset of LLMs were trained on a fixed training set, enabling us to dissociate model size from architecture and training set size. We used electrocorticography (ECoG) to measure neural activity in epilepsy patients while they listened to a 30 min naturalistic audio story. We fit electrode-wise encoding models using contextual embeddings extracted from each hidden layer of the LLMs to predict word-level neural signals. In line with prior work, we found that larger LLMs better capture the structure of natural language and better predict neural activity. We also found a logarithmic relationship where the encoding performance peaks in relatively earlier layers as model size increases. We also observed variations in the best-performing layer across different brain regions, corresponding to an organized language processing hierarchy.
Reproduced under the paper's license (CC BY), from the paper cited above.
Repositories
Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.
hassonlab/247-pickling
b7a6fcb060ecb8276b5dcb090b97e6f5b2983558, 1 March 2023Availability: 1 check, the latest on 26 September 2026: the link answers
- 26 September 2026: the link answers
18 files
- misc/
gpt2_test.ipynb , Jupyter, 201 lines - scripts/
electrode_utils.py , Python, 91 lines - scripts/
tfsemb_LMBase.py , Python, 71 lines - scripts/
tfsemb_concat.py , Python, 145 lines - scripts/
tfsemb_config.py , Python, 131 lines - scripts/
tfsemb_download.py , Python, 269 lines, 2 matches - scripts/
tfsemb_main.py , Python, 658 lines - scripts/
tfsemb_parser.py , Python, 45 lines - scripts/
tfspkl_build_matrices.py , Python, 260 lines - scripts/
tfspkl_config.py , Python, 118 lines - scripts/
tfspkl_main.py , Python, 392 lines - scripts/
tfspkl_parser.py , Python, 48 lines - scripts/
tfspkl_utils.py , Python, 258 lines - scripts/
utils.py , Python, 149 lines - submit.sh, Shell, 42 lines
- tests/
__init__.py , Python, 1 line - tests/
tests.py , Python, 35 lines - README.rst, Text, 18 lines
hassonlab/247-encoding
48c27e61a9fd8629d3f5da840d9f1ec2882a365f, 4 September 2026Availability: 1 check, the latest on 26 September 2026: the link answers
- 26 September 2026: the link answers
11 files
- scripts/
tfsenc_concat.py , Python, 45 lines - scripts/
tfsenc_config.py , Python, 167 lines, 1 match - scripts/
tfsenc_encoding.py , Python, 231 lines, 1 match - scripts/
tfsenc_load_signal.py , Python, 106 lines - scripts/
tfsenc_main.py , Python, 203 lines - scripts/
tfsenc_read_datum.py , Python, 415 lines - scripts/
tfserp_main.py , Python, 137 lines - scripts/
utils.py , Python, 38 lines - submit1.sh, Shell, 38 lines
- LICENSE, License, 21 lines
- README.md, Text, 3 lines
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 26 scripts, each with its path and the digest of its content;
- 4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Datasets cited
- hassonlab.github.io/
podcast-ecog-tutorials , at hassonlab.github.io; found in “Data availability”
Data availability
We have recently made the data publicly available (Zada et al., 2025). We have also provided tutorials for preprocessing the data and training encoding models: https://
The following previously published dataset was used:
Zada Z, Nastase SA, Aubrey B, Jalon I, Goldstein A, Michelmann S, Wang H, Hasenfratz L, Doyle W, Friedman D, Dugan P, Melloni L, Devore S, Devinsky O, Flinker A, Hasson U. 2025. The "Podcast" ECoG dataset. OpenNeuro.
Reproduced under the paper's license (CC BY), from the paper cited above.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, pages, dates, 16 authors, 1 keyword, 6 MeSH terms, 4 funders, 32 references.
Cite
This paper
Hong, Z., Wang, H., Zada, Z., Gazula, H., Turner, D., Aubrey, B., Niekerken, L., Doyle, W., Devore, S., Dugan, P., Friedman, D., Devinsky, O., Flinker, A., Hasson, U., Nastase, S., & Goldstein, A. Y. (2026). Larger language models better align with neural representations of natural language. eLife, 13, RP101204. https://
BibTeX
@article{hong2026larger,
author = {Hong, Zhuoqiao and Wang, Haocheng and Zada, Zaid and Gazula, Harshvardhan and Turner, David and Aubrey, Bobbi and Niekerken, Leonard and Doyle, Werner and Devore, Sasha and Dugan, Patricia and Friedman, Daniel and Devinsky, Orrin and Flinker, Adeen and Hasson, Uri and Nastase, Samuel and Goldstein, Ariel Y},
title = {{Larger language models better align with neural representations of natural language}},
journal = {eLife},
year = {2026},
month = sep,
volume = {13},
pages = {RP101204},
publisher = {eLife Sciences Publications, Ltd},
issn = {2050-084X},
doi = {10.7554/
url = {https://
pmid = {42746838},
pmcid = {PMC13581289}
}
RIS
TY - JOUR
AU - Hong, Zhuoqiao
AU - Wang, Haocheng
AU - Zada, Zaid
AU - Gazula, Harshvardhan
AU - Turner, David
AU - Aubrey, Bobbi
AU - Niekerken, Leonard
AU - Doyle, Werner
AU - Devore, Sasha
AU - Dugan, Patricia
AU - Friedman, Daniel
AU - Devinsky, Orrin
AU - Flinker, Adeen
AU - Hasson, Uri
AU - Nastase, Samuel
AU - Goldstein, Ariel Y
TI - Larger language models better align with neural representations of natural language
T2 - eLife
J2 - eLife
PY - 2026
DA - 2026/
VL - 13
SP - RP101204
SN - 2050-084X
PB - eLife Sciences Publications, Ltd
DO - 10.7554/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.7554/
"type": "article-journal",
"title": "Larger language models better align with neural representations of natural language",
"container-title": "eLife",
"author": [
{
"family": "Hong",
"given": "Zhuoqiao"
},
{
"family": "Wang",
"given": "Haocheng"
},
{
"family": "Zada",
"given": "Zaid"
},
{
"family": "Gazula",
"given": "Harshvardhan"
},
{
"family": "Turner",
"given": "David"
},
{
"family": "Aubrey",
"given": "Bobbi"
},
{
"family": "Niekerken",
"given": "Leonard"
},
{
"family": "Doyle",
"given": "Werner"
},
{
"family": "Devore",
"given": "Sasha"
},
{
"family": "Dugan",
"given": "Patricia"
},
{
"family": "Friedman",
"given": "Daniel"
},
{
"family": "Devinsky",
"given": "Orrin"
},
{
"family": "Flinker",
"given": "Adeen"
},
{
"family": "Hasson",
"given": "Uri"
},
{
"family": "Nastase",
"given": "Samuel"
},
{
"family": "Goldstein",
"given": "Ariel Y"
}
],
"container-title-short":
"volume": "13",
"page": "RP101204",
"DOI": "10.7554/
"PMID": "42746838",
"PMCID": "PMC13581289",
"ISSN": "2050-084X",
"publisher": "eLife Sciences Publications, Ltd",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
9,
16
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1038/s41467-024-46631-y [code]
- Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns.Journal: Nature communicationsIn common: scikit-learn, pandas, SciPy, 1 other tool, 11 references, 3 authors
- [2] doi:10.1038/s41597-025-05462-2 [code]
- The "Podcast" ECoG dataset for modeling neural activity during natural language comprehension.Journal: Scientific dataIn common: PyTorch, scikit-learn, pandas, 2 other tools, hassonlab.github.io/podcast-ecog-tutorials, intracranial EEG (iEEG / ECoG / SEEG), 9 references, 2 authors
- [3] doi:10.1038/s41562-025-02105-9 [code]
- A unified acoustic-to-speech-to-la
nguage embedding space captures the neural basis of natural language processing in everyday conversations. Journal: Nature human behaviourIn common: Hugging Face Transformers, Numba, PyTorch, 4 other tools, intracranial EEG (iEEG / ECoG / SEEG), 5 references, 3 authors - [4] doi:10.1038/s41467-024-49173-5 [code]
- Shared functional specialization in transformer-based language models and the human brain.Journal: Nature communicationsIn common: Hugging Face Transformers, PyTorch, scikit-learn, 3 other tools, 12 references, author Samuel Nastase
- [5] doi:10.1038/s41467-025-65518-0 [code]
- Temporal structure of natural language processing in the human brain corresponds to layered hierarchy of large language models.Journal: Nature communicationsIn common: Hugging Face Transformers, Numba, PyTorch, 4 other tools, intracranial EEG (iEEG / ECoG / SEEG), 7 references, 2 authors
- [6] doi:10.7554/elife.106543 [code]
- Stimulus dependencies-rather than next-word prediction-can explain pre-onset brain encoding in naturalistic listening designs.Journal: eLifeIn common: Hugging Face Transformers, PyTorch, scikit-learn, 3 other tools, 7 references
- [7] doi:10.1162/nol_a_00116 [code]
- Lexical-Semantic Content, Not Syntactic Structure, Is the Main Contributor to ANN-Brain Similarity of fMRI Responses in the Language Network.Journal: Neurobiology of language (Cambridge, Mass.)In common: Hugging Face Transformers, PyTorch, scikit-learn, 3 other tools, 6 references
- [8] doi:10.1038/s43588-025-00863-0 [code]
- Increasing alignment of large language models with language processing in the human brain.Journal: Nature computational scienceIn common: Hugging Face Transformers, PyTorch, scikit-learn, 3 other tools, 6 references
- [9] doi:10.1038/s42003-025-09377-x [code]
- Hierarchical linguistic predictions and cross-level information updating during narrative comprehension.Journal: Communications biologyIn common: Hugging Face Transformers, PyTorch, scikit-learn, 2 other tools, 3 references, author Adeen Flinker
- [10] doi:10.1162/imag.a.1227 [code]
- Large language models reveal the neural tracking of linguistic context in attended and unattended multi-talker speech.Journal: Imaging neuroscience (Cambridge, Mass.)In common: Hugging Face Transformers, PyTorch, scikit-learn, 3 other tools, intracranial EEG (iEEG / ECoG / SEEG), 5 references
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 2 repositories of the authors' code, each at its verified commit and with its license, 26 scripts, and 4 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:160b30025035fdd7…
Add the badge to the code's README
If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.
Markdown
<!-- oscr-badge:start -->
[](https://openscicode.org/badge/doi/10.7554/elife.101204)
<!-- oscr-badge:end -->reStructuredText
.. oscr-badge:start
.. image:: https://openscicode.org/badge/doi/10.7554/elife.101204.svg
:target: https://openscicode.org/badge/doi/10.7554/elife.101204
:alt: OSCR
.. oscr-badge:endHTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.7554/elife.101204"><img src="https://openscicode.org/badge/doi/10.7554/elife.101204.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".
If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.
Report a mismatch between the code and the paper · Report a code error (signed in)
Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.
Report a reproduction failure (signed in)
Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
