OSCR

Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games.

Overview

Authors: Pratheep Kumar Paranthaman1, Nikesh Bajaj2, Logan LaMont1
  1. Department of Computer Science, Elon University, Elon, NC 27244, USA
  2. School of Physical and Chemical Sciences, Queen Mary University, London E1 4NS, UK
Institutions: Elon University (United States); Queen Mary University of London (United Kingdom)
Journal: Sensors (Basel, Switzerland), volume 26, issue 12, article 3690
Dates: received 13 May 2026; accepted 6 June 2026; published online 10 June 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.3390/s26123690 · PMID 42356663 · PMCID PMC13307432 · OpenAlex W7164152237
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: EEG (modality), human (organism), cognitive (subfield)
Methods: Statistics, Smoothing, state filtering, decompositions
Keywords: mixed reality, extended reality games, electroencephalogram, interaction modalities, EEG analysis, NASA task load index, Self-Assessment Manikin, player experience
MeSH: Electroencephalography*, Video Games*, Adult, Brain, Female, Gestures, Humans, Male, Speech, User-Computer Interface, Virtual Reality (* major topic)
Topic: Augmented Reality Applications (Computer Vision and Pattern Recognition, Computer Science), according to OpenAlex
Citations: not cited yet (Europe PMC); 60 references in the paper

Abstract

Mixed reality (MR) technologies enable users to experience computer-generated content within the physical environment through spatial computing and head-mounted displays. By supporting real-time interaction through speech, gesture, gaze, and movement, MR offers new opportunities for game design beyond productivity and educational applications. However, relatively few studies have examined interaction modalities in MR games. In this paper, we present the design and deployment of four MR games on the Microsoft HoloLens 2: three that use monomodal input (speech, gaze, or gesture) and one that uses multimodal input (speech, gaze, and gesture). We conducted a study with ten participants and evaluated player experience using subjective self-reports of task load, emotional engagement, and comfort alongside objective measures, namely brain activity data collected with a five-channel electroencephalogram (EEG) device. Our preliminary findings suggest two clusters of interaction modalities based on subjective measures, a pattern that is also reflected in the objective EEG measures. Our analysis combining subjective and EEG data indicates that interaction modality influences task load and emotional engagement. Additionally, our functional connectivity analysis showed links in activity across the prefrontal, temporal, and occipital brain regions for different input modalities in the MR games.

Reproduced under the paper's license (CC BY), from the paper cited above.

Code

No file of the authors' code could be read here: it is described below, and read at its source.

spkit.github.io

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: the link answers
Software Heritage: not checked
Found in: the references
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
  • 27 September 2026: the link answers (HTTP 200)
At the source: spkit.github.io

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 0 scripts, each with its path and the digest of its content;
  • no match between paragraphs and code yet;
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

The data presented in this study are available upon request to the corresponding author. Availability of specific materials may vary depending on the nature of the request.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 3 authors, 8 keywords, 11 MeSH terms, 54 references.

Cite

This paper

Paranthaman, P. K., Bajaj, N., & LaMont, L. (2026). Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games. Sensors (Basel, Switzerland), 26(12), 3690. https://doi.org/10.3390/s26123690

BibTeX

@article{paranthaman2026electroencephalogram,
author = {Paranthaman, Pratheep Kumar and Bajaj, Nikesh and LaMont, Logan},
title = {{Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games}},
journal = {Sensors (Basel, Switzerland)},
year = {2026},
month = jun,
volume = {26},
number = {12},
pages = {3690},
publisher = {Multidisciplinary Digital Publishing Institute (MDPI)},
issn = {1424-8220},
doi = {10.3390/s26123690},
url = {https://doi.org/10.3390/s26123690},
pmid = {42356663},
pmcid = {PMC13307432}
}

RIS

TY - JOUR
AU - Paranthaman, Pratheep Kumar
AU - Bajaj, Nikesh
AU - LaMont, Logan
TI - Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games
T2 - Sensors (Basel, Switzerland)
J2 - Sensors (Basel)
PY - 2026
DA - 2026/06/10
VL - 26
IS - 12
SP - 3690
SN - 1424-8220
PB - Multidisciplinary Digital Publishing Institute (MDPI)
DO - 10.3390/s26123690
UR - https://doi.org/10.3390/s26123690
LA - en
ER -

CSL-JSON

{
"id": "10.3390/s26123690",
"type": "article-journal",
"title": "Electroencephalogram-Based Analysis of Monomodal and Multimodal Interaction in Mixed Reality Games",
"container-title": "Sensors (Basel, Switzerland)",
"author": [
{
"family": "Paranthaman",
"given": "Pratheep Kumar"
},
{
"family": "Bajaj",
"given": "Nikesh"
},
{
"family": "LaMont",
"given": "Logan"
}
],
"container-title-short": "Sensors (Basel)",
"volume": "26",
"issue": "12",
"page": "3690",
"DOI": "10.3390/s26123690",
"PMID": "42356663",
"PMCID": "PMC13307432",
"ISSN": "1424-8220",
"publisher": "Multidisciplinary Digital Publishing Institute (MDPI)",
"URL": "https://doi.org/10.3390/s26123690",
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
10
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.3389/fnrgo.2022.1007673 [code]
Predictions of task using neural modeling.
Journal: Frontiers in neuroergonomics
In common: EEG, cognitive, 2 references
[2] doi:10.3390/s26175327 [code]
Subject Identity Confounds qEEG Emotion Recognition on DEAP and DREAMER.
Journal: Sensors (Basel, Switzerland)
In common: EEG, cognitive, 2 references
[3] doi:10.3389/fnins.2022.744737 [code]
Assessment of instantaneous cognitive load imposed by educational multimedia using electroencephalography signals.
Journal: Frontiers in neuroscience
In common: EEG, cognitive, 2 references
[4] doi:10.3389/fncom.2022.1022787
Neuronal and behavioral affective perceptions of human and naturalness-reduced emotional prosodies.
Journal: Frontiers in computational neuroscience
In common: EEG, cognitive, 2 references
[5] doi:10.1016/j.eclinm.2024.102955
A telerehabilitation program to improve visual perception in children and adolescents with hemianopia consecutive to a brain tumor: a single-arm feasibility and proof-of-concept trial.
Journal: EClinicalMedicine
In common: 2 references
[6] doi:10.1038/s44271-025-00299-2 [code]
Awe is characterized as an ambivalent affect in the human behavior and cortex.
Journal: Communications psychology
In common: cognitive, 2 references
[7] doi:10.3389/fnins.2021.771533 [code]
Working With Environmental Noise and Noise-Cancelation: A Workload Assessment With EEG and Subjective Measures.
Journal: Frontiers in neuroscience
In common: EEG, 2 references
[8] doi:10.1162/imag.a.1134 [code]
Oscillatory and gaze signatures of socio-emotional speech processing, visuo-spatial cognition, and their interaction in a near-realistic dual-task MEG study.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: cognitive, 2 references
[9] doi:10.1038/s41598-025-23573-z [code]
Graph-theoretical analysis of EEG-based functional connectivity during emotional experience in virtual reality for emotion recognition.
Journal: Scientific reports
In common: EEG, cognitive, 1 reference
[10] doi:10.12688/f1000research.171170.2
A Review of Deep Learning Techniques for EEG-Based Emotion Recognition: Models, Methods, and Datasets.
Journal: F1000Research
In common: EEG, cognitive, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.3390/s26123690.svg)](https://openscicode.org/badge/doi/10.3390/s26123690)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.3390/s26123690.svg
   :target: https://openscicode.org/badge/doi/10.3390/s26123690
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.3390/s26123690"><img src="https://openscicode.org/badge/doi/10.3390/s26123690.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.