Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games.

Overview

1. Department of Computer Science, Elon University, Elon, NC 27244, USA
2. School of Physical and Chemical Sciences, Queen Mary University, London E1 4NS, UK

Abstract

Mixed reality (MR) technologies enable users to experience computer-generated content within the physical environment through spatial computing and head-mounted displays. By supporting real-time interaction through speech, gesture, gaze, and movement, MR offers new opportunities for game design beyond productivity and educational applications. However, relatively few studies have examined interaction modalities in MR games. In this paper, we present the design and deployment of four MR games on the Microsoft HoloLens 2: three that use monomodal input (speech, gaze, or gesture) and one that uses multimodal input (speech, gaze, and gesture). We conducted a study with ten participants and evaluated player experience using subjective self-reports of task load, emotional engagement, and comfort alongside objective measures, namely brain activity data collected with a five-channel electroencephalogram (EEG) device. Our preliminary findings suggest two clusters of interaction modalities based on subjective measures, a pattern that is also reflected in the objective EEG measures. Our analysis combining subjective and EEG data indicates that interaction modality influences task load and emotional engagement. Additionally, our functional connectivity analysis showed links in activity across the prefrontal, temporal, and occipital brain regions for different input modalities in the MR games.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

No file of the authors' code could be read here: it is described below, and read at its source.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

• 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
• 0 scripts, each with its path and the digest of its content;
• no match between paragraphs and code yet;
• neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

The data presented in this study are available upon request to the corresponding author. Availability of specific materials may vary depending on the nature of the request.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 3 authors, 8 keywords, 11 MeSH terms, 54 references.

Cite

This paper

Paranthaman, P. K., Bajaj, N., & LaMont, L. (2026). Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games. Sensors (Basel, Switzerland), 26(12), 3690. https://doi.org/10.3390/s26123690

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.3390/s26175327 [code]

Subject Identity Confounds qEEG Emotion Recognition on DEAP and DREAMER.

Journal: Sensors (Basel, Switzerland)

In common: EEG, cognitive, 2 references

[2] doi:10.1038/s42003-026-10394-7 [code]

Cognitive load weakens neural speech tracking without altering response timing.

Journal: Communications biology

In common: EEG, cognitive, 1 reference

[3] doi:10.1016/j.ynirp.2026.100347

App-based mindfulness meditation training enhances cognitive flexibility and modulates ACC and medial frontal gyrus activation during task switching in adolescent OCD.

Journal: Neuroimage. Reports

In common: EEG, cognitive, 1 reference

[4] doi:10.3390/bs16081364

Morphological Buffering in Neuro-Architecture: Interactive Effects of Color, Shape, and Area Proportion on Affective Processing.

Journal: Behavioral sciences (Basel, Switzerland)

In common: EEG, cognitive, 1 reference

[5] doi:10.3758/s13415-026-01446-w

People, places, and things: The impact of scene animacy on emotional modulation of the early posterior negativity.

Journal: Cognitive, affective & behavioral neuroscience

In common: EEG, cognitive, 1 reference

[6] doi:10.1111/psyp.70318

EEG Based Decoding of the Perception and Regulation of Taboo Words.

Journal: Psychophysiology

In common: EEG, cognitive, 1 reference

[7] doi:10.1186/s40708-026-00300-6 [code]

Anatomical-connectivity-guided functional connectivity reveals task-relevant pathways during proactive task-switching via recurrent graph neural networks.

Journal: Brain informatics

In common: EEG, cognitive, 1 reference

[8] doi:10.1371/journal.pone.0354510 [code]

Personalized adaptive virtual reality experience driven by electroencephalography-based pain recognition.

Journal: PloS one

In common: EEG, 1 reference

[9] doi:10.3389/fnhum.2026.1807535

The ASME-speller: 30-class auditory brain-computer interface speller using stream segregation and the QWERTY layout.

Journal: Frontiers in human neuroscience

In common: EEG, 1 reference

[10] doi:10.1371/journal.pone.0348608

A novel approach for the EEG-driven assessment of divided attention through mutual information theory: A case study at the wheel.

Journal: PloS one

In common: EEG, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.