OSCR

MAME: Multidimensional adaptive metamer exploration with human perceptual feedback.

Code ↔ Paper

3 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 3 matches
  1. [1] § Methods › Definition of the exploration direction of human metameric space ↔ abx_app/AttackCNN/utils/ica.py, lines 18–164 · score 0.58 · unit variance, independent component, mixing, whitened, transformation, ICA
  2. [2] § Methods › Definition of the exploration direction of human metameric space ↔ abx_app/AttackCNN/utils/ica.py, lines 18–164 · score 0.57 · unit variance, independent component, mixing, zero, ICA, matrix
  3. [3] § Methods › Generation of target images ↔ abx_app/AttackCNN/utils/pgd_attack.py, lines 18–98 · score 0.57 · torch.optim.Adam, PyTorch, optimization, layer

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 164 lines · 6.5 KB · MIT · 2 matches

  1. import copy
  2. from pathlib import Path
  3. import yaml
  6. import joblib
  7. import matplotlib.pyplot as plt
  8. import numpy as np
  9. from sklearn.decomposition import FastICA
  10. import torch
  12. from .. import config
  13. from .decomposition_handler import DecompositionHandler, V
  15. device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
  18. class ICAHandler(DecompositionHandler):
  19. ica_result: FastICA
  20. components_torch: torch.Tensor
  21. mean_torch: torch.Tensor
  22. mixing_torch: torch.Tensor
  24. def __init__(self, ica_model_file: Path | None):
  25. self.ica_model_file = ica_model_file
  26. self.ica_result: FastICA
  28. if ica_model_file and ica_model_file.exists():
  29. loaded_data = joblib.load(ica_model_file)
  30. self.ica_result = loaded_data["ica_result"]
  31. self.components_torch = torch.tensor(self.ica_result.components_, dtype=torch.float32, device=device)
  32. self.mean_torch = torch.tensor(self.ica_result.mean_, dtype=torch.float32, device=device)
  33. self.mixing_torch = torch.tensor(self.ica_result.mixing_, dtype=torch.float32, device=device)
  35. def perform_ica(self, data_file, n_components=None, data=None, random_seed=42):
  36. if data is None:
  37. if data_file and data_file.exists():
  38. data = np.load(data_file, allow_pickle=True)
  39. else:
  40. raise ValueError("Gram matrix data is not provided and data_file is None or does not exists")
  42. ica = FastICA(
  43. n_components=n_components, whiten="unit-variance", max_iter=500, tol=1e-3, random_state=random_seed
  44. )
  45. ica.fit(data)
  46. self.ica_result = ica
  47. self.components_torch = torch.tensor(self.ica_result.components_, dtype=torch.float32, device=device)
  48. self.mean_torch = torch.tensor(self.ica_result.mean_, dtype=torch.float32, device=device)
  49. self.mixing_torch = torch.tensor(self.ica_result.mixing_, dtype=torch.float32, device=device)
  51. if self.ica_model_file:
  52. model_data = {"ica_result": ica}
  53. joblib.dump(model_data, self.ica_model_file)
  55. return self.ica_result
  57. def transform_coordinate(self, data_flat: V):
  58. if self.ica_result is None:
  59. raise ValueError("ICAResult is not initialized. Please perform ICA first.")
  60. if isinstance(data_flat, np.ndarray):
  61. return self.ica_result.transform(data_flat)
  62. elif isinstance(data_flat, torch.Tensor):
  63. data_flat -= self.mean_torch
  64. return torch.matmul(data_flat, self.components_torch.T)
  65. else:
  66. raise ValueError("Input must be np.ndarray or torch.Tensor")
  68. def inverse_coordinate(self, coordinates: V):
  69. if self.ica_result is None:
  70. raise ValueError("ICAResult is not initialized. Please perform ICA first.")
  71. if isinstance(coordinates, np.ndarray):
  72. return self.ica_result.inverse_transform(coordinates)
  73. elif isinstance(coordinates, torch.Tensor):
  74. coordinates = torch.matmul(coordinates, self.mixing_torch.T)
  75. return coordinates + self.mean_torch
  77. def extract_explained_variance(self, train_data_file):
  78. if train_data_file and train_data_file.exists():
  79. X = np.load(train_data_file, allow_pickle=True)
  80. else:
  81. raise ValueError("Gram matrix data is not provided and data_file is None or does not exists")
  82. S = self.ica_result.transform(X)
  83. X_centered = X - self.ica_result.mean_
  84. A = self.ica_result.mixing_
  86. explained_variance_ratios = []
  88. for i in range(S.shape[1]):
  89. S_i = np.zeros_like(S)
  90. S_i[:, i] = S[:, i]
  92. X_hat_i = np.dot(S_i, A.T)
  94. explained_variance_i = (
  95. 1 - np.linalg.norm(X_centered - X_hat_i, ord="fro") ** 2 / np.linalg.norm(X_centered, ord="fro") ** 2
  96. )
  97. explained_variance_ratios.append(explained_variance_i)
  99. sorted_indices = np.argsort(explained_variance_ratios)[::-1]
  100. sorted_ratios = np.array(explained_variance_ratios)[sorted_indices]
  102. return sorted_ratios, sorted_indices
  104. def plot_explained_variance(self, explained_variance_ratios, layer_name):
  105. fig, ax1 = plt.subplots(figsize=(10, 6))
  107. indices = np.arange(1, len(explained_variance_ratios) + 1)
  109. ax1.bar(
  110. indices,
  111. explained_variance_ratios,
  112. color="m",
  113. width=0.8,
  114. alpha=0.7,
  115. edgecolor="black",
  116. label="Contribution Ratio",
  117. )
  118. ax1.set_xlabel("Independent Components", fontsize=42)
  119. ax1.set_ylabel("Contribution Ratio", color="m", fontsize=42)
  120. ax1.tick_params(axis="x", labelsize=32)
  121. ax1.tick_params(axis="y", labelcolor="m", labelsize=32)
  122. ax1.set_ylim(0, max(explained_variance_ratios) * 1.1)
  124. ax1.grid(True, which="both", linestyle="--", linewidth=0.5, alpha=0.7)
  125. fig.tight_layout()
  127. plt.title(f"Explained Variance of {layer_name}", fontsize=42)
  128. title = Path(config["ica"]["ica_explained_variance_fig"]) / f"{layer_name}.png"
  129. title.parent.mkdir(parents=True, exist_ok=True)
  130. plt.savefig(
  131. title,
  132. bbox_inches="tight",
  133. )
  135. def change_model_top_k_components(self, sorted_indices, top_k, ica_model_file):
  136. if not hasattr(self.ica_result, "components_"):
  137. raise ValueError("ICAResult is not initialized or invalid. Please perform ICA first.")
  139. top_indices = sorted_indices[:top_k]
  141. ica_top_k = copy.deepcopy(self.ica_result)
  143. ica_top_k.components_ = self.ica_result.components_[top_indices, :]
  144. ica_top_k.mixing_ = self.ica_result.mixing_[:, top_indices]
  145. ica_top_k.n_components = top_k # type: ignore
  147. model_data = {"ica_result": ica_top_k}
  148. joblib.dump(model_data, ica_model_file)
  150. def change_model_selected_components(self, selected_indices, ica_model_file):
  151. if not hasattr(self.ica_result, "components_"):
  152. raise ValueError("ICAResult is not initialized or invalid. Please perform ICA first.")
  154. ica_selected = copy.deepcopy(self.ica_result)
  156. ica_selected.components_ = self.ica_result.components_[selected_indices, :]
  157. ica_selected.mixing_ = self.ica_result.mixing_[:, selected_indices]
  158. ica_selected.n_components = len(selected_indices) # type: ignore
  160. # Save the updated ICA model
  161. model_data = {"ica_result": ica_selected}
  162. joblib.dump(model_data, ica_model_file)
  164. print(f"Saved updated ICA model with selected components to {ica_model_file}")

ica.py at commit f8750ec, under MIT · at the source

Overview

Authors: Mina Kamao1, Hayato Ono1, Ayumu Yamashita2,1, Kaoru Amano1, Masataka Sawayama3,4,1
  1. Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan
  2. Graduate School of System Informatics, Kobe University, Hyogo, Japan
  3. Graduate School of Information Science and Technology, Hokkaido University, Hokkaido, Japan
  4. Prometech CG Research, Tokyo, Japan
Institutions: The University of Tokyo (Japan); Kobe University (Japan); Hokkaido University (Japan)
Journal: Journal of vision, volume 26, issue 8, article 1
Dates: received 17 September 2025; accepted 22 June 2026; published online 3 August 2026; in print August 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1167/jov.26.8.1 · PMID 42545067 · PMCID PMC13440621 · OpenAlex W7172329423
Open access: gold, a free copy (OpenAlex)
Preprint: arxiv:2503.13212
Status: code verified
Categories: behavior only (modality), human (organism), cognitive (subfield)
Methods: Smoothing, state filtering, decompositions
Keywords: metamers, texture synthesis, psychophysics, cognitive neuroscience, machine learning
MeSH: Feedback, Sensory*, Neural Networks, Computer*, Visual Perception*, Convolutional Neural Networks, Female, Humans, Photic Stimulation, Psychophysics (* major topic)
Topic: Face Recognition and Perception (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Citations: not cited yet (Europe PMC); 45 references in the paper

Abstract

Alignment between human brain networks and artificial models has become an active research area in vision science and machine learning. A widely adopted approach is identifying “metamers,” stimuli that are physically different yet perceptually equivalent within a system. However, conventional methods lack a direct approach to searching for the human metameric space. Instead, researchers first develop biologically inspired models and then infer about human metamers indirectly by testing whether model metamers also appear as metamers to humans. Here, we propose the multidimensional adaptive metamer exploration (MAME) framework, enabling direct, high-dimensional exploration of human metameric spaces through online image generation guided by human perceptual feedback. MAME modulates reference images across multiple dimensions based on hierarchical neural network responses, adaptively updating generation parameters according to participants’ perceptual discriminability. Using MAME, we successfully measured multidimensional human metameric spaces within a single psychophysical experiment. Experimental results using a biologically plausible convolutional neural network (CNN) model showed that human discrimination sensitivity was lower for metameric images based on Gram-matrix representations derived from low-level CNN features than for those derived from high-level CNN features. The finding suggests a relatively worse alignment between the metameric spaces of humans and the CNN model for low-level processing compared with high-level processing. Counterintuitively, given recent discussions on alignment at higher representational levels, our results highlight the importance of early visual computations in shaping biologically plausible models. Our MAME framework can serve as a future scientific tool for directly investigating the functional organization of human vision.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 3 matches between paragraphs and lines of code.

Zenodo 20789863

License: MIT
State: the link answers, verified on 26 September 2026
Evidence: files inventoried
Size: 1 file
Software Heritage: not checked
Found in: the text, “Proposed approach: The MAME framework”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (27 files), PyTorch (27 files), Matplotlib (14 files), pandas (4 files), SciPy (3 files), Pillow (2 files), scikit-learn (2 files), OpenCV (1 file), psignifit (1 file)
Availability: 1 check, the latest on 26 September 2026: the link answers (HTTP 200)
  • 26 September 2026: the link answers (HTTP 200)
189 files
At the source: doi.org/10.5281/zenodo.20789863

amano-k-lab/mame_hil

License: MIT
State: the link answers, verified on 26 September 2026
Evidence: files inventoried
Commit: f8750eca778f8f3c90c7a0db7ba6e5970e71d0a5, 22 June 2026
Languages: JavaScript (105), Python (80), Jupyter (2)
Size: 247 files, 187 scripts
Software Heritage: not archived
Found in: the text, “Proposed approach: The MAME framework”
Holds: README, license file, environment (requirements.txt), 2 notebooks
Not found: CITATION.cff, tests, continuous integration, documentation
Tools: NumPy (27 files), PyTorch (27 files), Matplotlib (14 files), pandas (4 files), SciPy (3 files), Pillow (2 files), scikit-learn (2 files), OpenCV (1 file), psignifit (1 file)
Availability: 1 check, the latest on 26 September 2026: the link answers
  • 26 September 2026: the link answers
189 files
At the source: github.com/amano-k-lab/mame_hil

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 374 scripts, each with its path and the digest of its content;
  • 3 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

  • osf:3tfhu, at OSF; found in the text, “Proposed approach: The MAME framework”

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 5 authors, 5 keywords, 8 MeSH terms, 22 references.

Cite

This paper

Kamao, M., Ono, H., Yamashita, A., Amano, K., & Sawayama, M. (2026). MAME: Multidimensional adaptive metamer exploration with human perceptual feedback. Journal of vision, 26(8), 1. https://doi.org/10.1167/jov.26.8.1

BibTeX

@article{kamao2026mame,
author = {Kamao, Mina and Ono, Hayato and Yamashita, Ayumu and Amano, Kaoru and Sawayama, Masataka},
title = {{MAME: Multidimensional adaptive metamer exploration with human perceptual feedback}},
journal = {Journal of vision},
year = {2026},
month = aug,
volume = {26},
number = {8},
pages = {1},
publisher = {Association for Research in Vision and Ophthalmology},
issn = {1534-7362},
doi = {10.1167/jov.26.8.1},
url = {https://doi.org/10.1167/jov.26.8.1},
pmid = {42545067},
pmcid = {PMC13440621}
}

RIS

TY - JOUR
AU - Kamao, Mina
AU - Ono, Hayato
AU - Yamashita, Ayumu
AU - Amano, Kaoru
AU - Sawayama, Masataka
TI - MAME: Multidimensional adaptive metamer exploration with human perceptual feedback
T2 - Journal of vision
J2 - J Vis
PY - 2026
DA - 2026/08/01
VL - 26
IS - 8
SP - 1
SN - 1534-7362
PB - Association for Research in Vision and Ophthalmology
DO - 10.1167/jov.26.8.1
UR - https://doi.org/10.1167/jov.26.8.1
LA - en
ER -

CSL-JSON

{
"id": "10.1167/jov.26.8.1",
"type": "article-journal",
"title": "MAME: Multidimensional adaptive metamer exploration with human perceptual feedback",
"container-title": "Journal of vision",
"author": [
{
"family": "Kamao",
"given": "Mina"
},
{
"family": "Ono",
"given": "Hayato"
},
{
"family": "Yamashita",
"given": "Ayumu"
},
{
"family": "Amano",
"given": "Kaoru"
},
{
"family": "Sawayama",
"given": "Masataka"
}
],
"container-title-short": "J Vis",
"volume": "26",
"issue": "8",
"page": "1",
"DOI": "10.1167/jov.26.8.1",
"PMID": "42545067",
"PMCID": "PMC13440621",
"ISSN": "1534-7362",
"publisher": "Association for Research in Vision and Ophthalmology",
"URL": "https://doi.org/10.1167/jov.26.8.1",
"language": "en",
"issued": {
"date-parts": [
[
2026,
8,
1
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1162/imag.a.1207 [code]
Investigating the temporal dynamics and modeling of mid-level feature representations in humans.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: OpenCV, Pillow, PyTorch, 5 other tools, cognitive, 3 references
[2] doi:10.1038/s41598-026-53152-9 [code]
Layer-specific feature preference in a trained AlexNet model revealed by stylized image analysis.
Journal: Scientific reports
In common: OpenCV, Pillow, PyTorch, 3 other tools, 3 references
[3] doi:10.7554/elife.105953 [code]
Top-down feedback in deep neural networks leads to functional differences during audiovisual integration.
Journal: eLife
In common: Pillow, PyTorch, scikit-learn, 4 other tools, 2 references
[4] doi:10.1038/s41467-026-76098-y [code]
A single computational objective can produce specialization of streams in visual cortex.
Journal: Nature communications
In common: Pillow, PyTorch, scikit-learn, 4 other tools, 2 references
[5] doi:10.1371/journal.pcbi.1014263 [code]
MIRAGE: Robust multi-modal architectures translate fMRI-to-image models from vision to mental imagery.
Journal: PLoS computational biology
In common: OpenCV, Pillow, PyTorch, 5 other tools, 1 reference
[6] doi:10.1016/j.isci.2026.116168 [code]
See the small lesions: Frequency-guided spatial debiasing GAN for multimodal medical image fusion.
Journal: iScience
In common: OpenCV, Pillow, PyTorch, 5 other tools, 1 reference
[7] doi:10.1038/s41467-026-74358-5 [code]
Brain-inspired spatial intelligence for embodied agents.
Journal: Nature communications
In common: OpenCV, Pillow, PyTorch, 5 other tools, cognitive
[8] doi:10.1167/jov.26.5.7 [code]
Representations in vision and language converge in a shared, multidimensional space of perceived similarities.
Journal: Journal of vision
In common: OpenCV, Pillow, PyTorch, 5 other tools, cognitive
[9] doi:10.7554/elife.107933 [code]
Modality-agnostic decoding of vision and language from fMRI.
Journal: eLife
In common: OpenCV, Pillow, PyTorch, 5 other tools, cognitive
[10] doi:10.1038/s41467-026-75653-x [code]
Concept2Brain: an AI model for predicting neurophysiological responses to text and pictures.
Journal: Nature communications
In common: Pillow, PyTorch, scikit-learn, 4 other tools, cognitive, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.