Bidirectional effects of speech and gesture: evidence from a novel levels-of-congruence paradigm.

Overview

1. Department of Language Science and Technology, Saarland University, Saarbrücken, Germany

Abstract

Introduction: Previous research has shown that speech and gesture are integrated during comprehension, with differences in response times and electrophysiological activity observed between incongruent and congruent gesture—speech pairings. Beyond these incongruency effects, it remains unclear whether comprehenders use gestures to adapt or extend conventional interpretations for novel communicative purposes.

Method: The current study examined the bidirectional influence of speech and gesture by employing a novel levels-of-congruence paradigm, in which the same metaphor (e.g., Men are oaks) was paired with a gesture depicting either its preferred interpretation (e.g., men are robust) or a less preferred but possible interpretation (e.g., men are tall). In Experiment 1, participants rated the meaningfulness of preferred and possible interpretations across three gesture conditions (gesture-of-preferred, gesture-of-possible, non-gestured). In Experiment 2, participants selected a single interpretation under the same conditions. In Experiment 3, participants viewed clips of an actor producing the same gesture and utterance combination (e.g., flexing both arms while saying Men are oaks) in one of three multimodal conditions: no additional cues (muted, blurred lips), only lip movements (muted, visible lips), or audible speech. Then they rated how well the gesture matched the gestured interpretation (e.g., robust), an alternative interpretation consistent with the speech but not the gesture (e.g., tall), and an unrelated interpretation (e.g., hikers).

Results: In Experiments 1 and 2, participants consistently rated the preferred interpretation as the most meaningful and selected it most often across all gesture conditions. However, gestures depicting the possible interpretation significantly increased both the meaningfulness ratings and the likelihood of being selected of the possible interpretation. In Experiment 3, the presence of speech cues significantly increased ratings for the alternative interpretation compared with the condition without additional cues.

Discussion: These findings demonstrate bidirectional interactions between speech and gesture during metaphor comprehension. Gestures constrained metaphor interpretation, whereas speech broadened the range of interpretations attributed to gestures. Overall, the results support the integrated-systems hypothesis by showing that speech and gesture jointly contribute to the construction of meaning.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

No file of the authors' code could be read here: it is described below, and read at its source.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

• 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
• 0 scripts, each with its path and the digest of its content;
• no match between paragraphs and code yet;
• neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary material. The online repository can be accessed here: https://osf.io/cfk7j.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

• Authors: added Laura Pissani (0000-0003-2884-4515); Vera Demberg (0000-0002-8834-0020); removed Laura Pissani; Vera Demberg

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 2 authors, 5 keywords, 1 funder, 34 references.

Cite

This paper

Pissani, L., & Demberg, V. (2026). Bidirectional effects of speech and gesture: evidence from a novel levels-of-congruence paradigm. Frontiers in psychology, 17, 1851588. https://doi.org/10.3389/fpsyg.2026.1851588

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1162/opmi.a.351

Child-Directed Speech Facilitates Semantic Role Learning: A Machine Learning Approach.

Journal: Open mind : discoveries in cognitive science

In common: cognitive, 3 references

[2] doi:10.1016/j.bandl.2026.105833 [code]

Neural tracking of surprisal in Spanish-English bilingual children during naturalistic heritage language listening.

Journal: Brain and language

In common: cognitive, 1 reference

[3] doi:10.1038/s41586-026-10691-5 [code]

Mapping the neuronal building blocks of human language with language models.

Journal: Nature

In common: cognitive, 1 reference

[4] doi:10.1038/s41598-026-56170-9 [code]

Path integration from optic flow and the role of eye movements.

Journal: Scientific reports

In common: cognitive, 1 reference

[5] doi:10.1186/s13229-026-00730-3 [code]

Predicting emotional valence in autism: a preregistered study in the Bayesian Brain framework.

Journal: Molecular autism

In common: cognitive, 1 reference

[6] doi:10.1162/nol.a.266 [code]

The Role of the Anterior Temporal Lobe in Reading: An HD-tDCS Study.

Journal: Neurobiology of language (Cambridge, Mass.)

In common: cognitive, 1 reference

[7] doi:10.1371/journal.pbio.3003899 [code]

Social interactions between people of same and different generations shape longitudinal changes in interpersonal neural synchrony, loneliness, and social connection.

Journal: PLoS biology

In common: 1 reference

[8] doi:10.1038/s41398-026-04172-6 [code]

Obesity is associated with greater variability of reward signals in the nucleus accumbens.

Journal: Translational psychiatry

In common: 1 reference

[9] doi:10.1162/opmi.a.372 [code]

Broadening the Agent Preference Hypothesis Through Experiencers: Eye-Tracking and EEG Evidence of Proto-Agents and Proto-Patients.

Journal: Open mind : discoveries in cognitive science

In common: 1 reference

[10] doi:10.1111/infa.70114 [code]

Statistics in Motion: Does the Infant Motor System Predict Actions Based on Their Transitional Probability?

Journal: Infancy : the official journal of the International Society on Infant Studies

In common: 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.