OSCR

Redundancy maximization as a principle of associative memory learning in Hopfield networks.

Code ↔ Paper

4 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 4 matches
  1. [1] § Results › Performance comparison ↔ paper_hopfield/src/eval_rule.py, lines 32 to 71 · score 0.61 · minimum probability flow, Descent L2, Tolmachev, MPF, Hopfield · Report a mismatch
  2. [2] § Results › Mutual information goals ↔ paper_hopfield/plotting/Figure_3.ipynb, lines 37 to 69 · score 0.55 · classical goal functions, co information, mutual information, networks trained, capacity, redundancy · Report a mismatch
  3. [3] § Results › Performance comparison ↔ paper_hopfield/src/eval_rule.py, lines 32 to 71 · score 0.54 · minimum probability flow, descent L2, Tolmachev, MPF · Report a mismatch
  4. [4] § Methods › Information-theoretic learning ↔ paper_hopfield/src/hopfield.py, lines 157 to 267 · score 0.53 · firing probability, activation function, discrete, binning, training, PID · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 109 lines · 4.1 KB · BSD-3-Clause · 2 matches

  1. import logging
  2. import os
  3. import sys
  4. #external packages
  5. import fastprogress
  6. from functools import partial
  7. import numpy as np
  8. import hydra
  9. import torch
  10. from torch import nn
  11. #custom packages
  12. from pathlib import Path
  13. sys.path.append(str(Path(__file__).parents[1]))
  14. sys.path.append(str(Path(__file__).parents[2]))
  15. from src import hopfield, training, tolmachev, mpf
  16. from im_net import datamanager, im_model, plotting, datasets
  17. import im_net.helper_functions as hf
  19. log = logging.getLogger(__name__)
  21. def test(dataloader, model:hopfield.Hopfield,device,noise:float=0.):
  22. model.eval()
  23. sim_function=nn.CosineSimilarity(1)
  24. epoch_acc=[]
  25. for batch in dataloader:
  26. noisy_batch:torch.Tensor = training.noisy_patterns(batch,noise)
  27. result = model.fixed_point(noisy_batch)
  28. similarity = torch.mean(sim_function(batch,result)).item()
  29. epoch_acc.append(similarity)
  30. return np.mean(epoch_acc)
  32. def choose_learning_rule(cfg):
  33. """
  34. Rules implemented: Hebb, DescentL2, Gardner, GardnerKM, Minimum Probability Flow
  35. Interface for learning rule: In the end only takes patterns and a dict **params.
  36. """
  37. rule_name = cfg.learning_rule.name
  38. match rule_name:
  39. case 'Hebb':
  40. learning_rule = hopfield.hebb
  41. case 'DescentL2':
  42. learning_rule = tolmachev.descent_l2
  43. #deal with bias
  44. if cfg.learning_rule.bias==True:
  45. bias = np.zeros((cfg.learning_rule.len))
  46. learning_rule = partial(learning_rule,biases=bias)
  47. initial_weight = np.zeros((cfg.learning_rule.len,cfg.learning_rule.len))
  48. learning_rule = partial(learning_rule,weights=initial_weight)
  49. case 'Gardner':
  50. learning_rule = tolmachev.Gardner
  51. #deal with bias
  52. bias = np.zeros((cfg.learning_rule.len))
  53. initial_weight = cfg.learning_rule.eps * np.ones((cfg.learning_rule.len,cfg.learning_rule.len))
  54. learning_rule = partial(learning_rule,weights=initial_weight,biases=bias)
  55. case 'GardnerKM':
  56. learning_rule = tolmachev.Gardner_Krauth_Mezard
  57. #deal with bias
  58. bias = np.zeros((cfg.learning_rule.len))
  59. initial_weight = cfg.learning_rule.eps * np.ones((cfg.learning_rule.len,cfg.learning_rule.len))
  60. learning_rule = partial(learning_rule,weights=initial_weight, biases=bias)
  61. case 'DescentExpSI':
  62. learning_rule = tolmachev.descent_exp_barrier_si
  63. #deal with bias
  64. bias = np.zeros((cfg.learning_rule.len))
  65. initial_weight = cfg.learning_rule.eps * np.ones((cfg.learning_rule.len,cfg.learning_rule.len))
  66. learning_rule = partial(learning_rule,weights=initial_weight, biases=bias)
  67. case 'mpf':
  68. learning_rule = mpf.binary_mpf
  69. case 'trivial':
  70. learning_rule = hopfield.trivial
  71. return learning_rule
  74. def learn_patterns(model:hopfield.BaseHopfield,dataset,learning_rule,rule_params):
  75. patterns = dataset[:]
  76. if rule_params.bias:
  77. weights,bias = learning_rule(patterns=patterns,**(rule_params.params or {}))
  78. model.set_weights(torch.tensor(weights),torch.tensor(bias))
  79. else:
  80. weights = learning_rule(patterns=patterns,**(rule_params.params or {}))
  81. model.set_weights(torch.tensor(weights))
  82. return model
  84. @hydra.main(config_path="../conf", config_name="eval_rule", version_base=None)
  85. def main(cfg):
  86. dm = training.init_run(cfg)
  87. device=hf.get_device(cfg.params.pref_gpu)
  88. log.info(f"Starting evaluation of learning rule.")
  90. learning_rule = choose_learning_rule(cfg)
  91. pattern_range=range(cfg.params.start,cfg.params.stop,cfg.params.step)
  92. progress_bar = fastprogress.progress_bar(pattern_range)
  93. dm.allocate_hdf(dset_names=['patterns','acc'], dset_length=len(pattern_range), group='capacity')
  94. for i,num_patterns in enumerate(progress_bar):
  95. #generate data
  96. dataset = datasets.HopfieldDataset(num_patterns,cfg.params.neurons)
  97. testloader = torch.utils.data.DataLoader(dataset,num_patterns,True)
  98. #train model
  99. model = hopfield.Hopfield(cfg.eval_model).to(device)
  100. learn_patterns(model,dataset,learning_rule,cfg.learning_rule)
  101. #eval
  102. acc = training.test(testloader,model,cfg.testing.noise,cfg.storage.testing.threshold_test,cfg.storage.testing.threshold)
  103. dm.write_to_dataset(['patterns','acc'],[num_patterns,acc],i,group='capacity')
  104. progress_bar.comment=f"patterns={num_patterns} : acc.={100*acc:.2f}% "
  105. log.info(f"Finished evaluating learning rule.")
  107. if __name__ == "__main__":
  108. main()
  109. #developing()

eval_rule.py at commit ecc3d7e, under BSD-3-Clause · at the source

Overview

Authors: Mark Blümel1, Andreas C Schneider1,2, Valentin Neuhaus1,2, David A Ehrlich1,2,3, Marcel Graetz4, Michael Wibral1,3, Abdullah Makkeh1,3, Viola Priesemann1,2
ORCID iDs: Andreas C Schneider, Valentin Neuhaus, David A Ehrlich, Michael Wibral, Abdullah Makkeh, Viola Priesemann
  1. Complex Systems Theory, Max Planck Institute for Dynamics and Self-Organization, Göttingen, Germany
  2. Faculty of Physics, Institute for the Dynamics of Complex Systems, University of Göttingen, Göttingen, Germany
  3. Göttingen Campus Institute for Dynamics of Biological Networks, University of Göttingen, Göttingen, Germany
  4. Champalimaud Centre for the Unknown, Lisbon, Portugal
Institutions: Max Planck Institute for Dynamics and Self-Organization (Germany); University of Göttingen (Germany); Göttingen Campus Institut für Dynamic biologischer Netzwerke (Germany); Champalimaud Foundation (Portugal)
Journal: Nature communications, volume 17, issue 1, article 11499
Dates: received 23 October 2025; accepted 29 September 2026; published online 6 November 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1038/s41467-026-78777-2 · PMID 42995199 · PMCID PMC13743752 · OpenAlex W7104000254
Open access: gold, a free copy (OpenAlex)
Preprint: arxiv:2511.02584
Status: code verified (how the code is found and verified)
Methods: Connectivity
Keywords: Computer science, Information theory and computation, Learning algorithms
Topic: Neural Networks and Applications (Artificial Intelligence, Computer Science), according to OpenAlex
Funding: Deutsche Forschungsgemeinschaft (German Research Foundation) (454648639, EXC 2067/1-390729940, 502807174); EC | EU Framework Programme for Research and Innovation H2020 | H2020 Priority Excellent Science | H2020 Future and Emerging Technologies (H2020 Excellent Science - Future and Emerging Technologies) (HBP-SP3.1-SGA1-T3.6.1)
Citations: not cited yet (Europe PMC); 34 references in the paper

Abstract

Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain incompletely understood. To formally characterize the local information processing in such systems, we employ a recent extension of information theory—Partial Information Decomposition. It decomposes the contribution of different inputs to an output into unique information from each input, redundant information across inputs, and synergistic information that emerges from combining different inputs. Applying this framework to individual neurons in classical Hopfield networks we find that below the memory capacity, the information in a neuron’s activity is characterized by high redundancy between the external pattern input and the internal recurrent input, while synergy and unique information are close to zero until the memory capacity is surpassed and performance drops steeply. Inspired by this observation, we use redundancy maximization at each neuron as an information-theoretic learning goal. This increases the network’s memory capacity to 1.59, a more than tenfold improvement over the 0.14 capacity of classical Hopfield networks, and also outperforming recent state-of-the-art implementations of Hopfield networks. Overall, this work establishes redundancy maximization as a design principle for associative memories and opens pathways for new associative memory models based on information-theoretic goals.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.

Priesemann-Group/Infomorphic_Networks

License: BSD-3-Clause
State: the link answers, verified on 9 November 2026
Evidence: files inventoried
Commit: ecc3d7e066049fac89a6a42c8ad37b382e0bd996, 28 August 2026
Languages: Python (21), Jupyter (16), Shell (2)
Size: 182 files, 39 scripts
Software Heritage: archived
Found in: “Code availability”
Holds: README, license file, environment (env.yml), 16 notebooks
Not found: CITATION.cff, tests, continuous integration, documentation
Tools: NumPy (31 files), Matplotlib (19 files), PyTorch (17 files), pandas (12 files), seaborn (10 files), SciPy (8 files), scikit-learn (4 files), h5py (1 file)
Availability: 1 check, the latest on 9 November 2026: the link answers
  • 9 November 2026: the link answers
41 files
At the source: github.com/Priesemann-Group/Infomorphic_Networks

Code availability

The code to reproduce the results is available via the public GitHub repository https://github.com/Priesemann-Group/Infomorphic_Networks.

Reproduced under the paper's license (CC BY), from the paper cited above.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 39 scripts, each with its path and the digest of its content;
  • 4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability Statement

The simulation data generated in this study can be reproduced using the code and parameters described in the Methods and Supplementary Information. The code is available at https://github.com/Priesemann-Group/Infomorphic_Networks. No external datasets were used in this study.

The code to reproduce the results is available via the public GitHub repository https://github.com/Priesemann-Group/Infomorphic_Networks.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 9 November 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 8 authors, 3 keywords, 2 funders, 19 references.

Cite

This paper

Blümel, M., Schneider, A. C., Neuhaus, V., Ehrlich, D. A., Graetz, M., Wibral, M., Makkeh, A., & Priesemann, V. (2026). Redundancy maximization as a principle of associative memory learning in Hopfield networks. Nature communications, 17(1), 11499. https://doi.org/10.1038/s41467-026-78777-2

BibTeX

@article{blumel2026redundancy,
author = {Blümel, Mark and Schneider, Andreas C and Neuhaus, Valentin and Ehrlich, David A and Graetz, Marcel and Wibral, Michael and Makkeh, Abdullah and Priesemann, Viola},
title = {{Redundancy maximization as a principle of associative memory learning in Hopfield networks}},
journal = {Nature communications},
year = {2026},
month = nov,
volume = {17},
number = {1},
pages = {11499},
publisher = {Nature Publishing Group},
issn = {2041-1723},
doi = {10.1038/s41467-026-78777-2},
url = {https://doi.org/10.1038/s41467-026-78777-2},
pmid = {42995199},
pmcid = {PMC13743752}
}

RIS

TY - JOUR
AU - Blümel, Mark
AU - Schneider, Andreas C
AU - Neuhaus, Valentin
AU - Ehrlich, David A
AU - Graetz, Marcel
AU - Wibral, Michael
AU - Makkeh, Abdullah
AU - Priesemann, Viola
TI - Redundancy maximization as a principle of associative memory learning in Hopfield networks
T2 - Nature communications
J2 - Nat Commun
PY - 2026
DA - 2026/11/06
VL - 17
IS - 1
SP - 11499
SN - 2041-1723
PB - Nature Publishing Group
DO - 10.1038/s41467-026-78777-2
UR - https://doi.org/10.1038/s41467-026-78777-2
LA - en
ER -

CSL-JSON

{
"id": "10.1038/s41467-026-78777-2",
"type": "article-journal",
"title": "Redundancy maximization as a principle of associative memory learning in Hopfield networks",
"container-title": "Nature communications",
"author": [
{
"family": "Blümel",
"given": "Mark"
},
{
"family": "Schneider",
"given": "Andreas C"
},
{
"family": "Neuhaus",
"given": "Valentin"
},
{
"family": "Ehrlich",
"given": "David A"
},
{
"family": "Graetz",
"given": "Marcel"
},
{
"family": "Wibral",
"given": "Michael"
},
{
"family": "Makkeh",
"given": "Abdullah"
},
{
"family": "Priesemann",
"given": "Viola"
}
],
"container-title-short": "Nat Commun",
"volume": "17",
"issue": "1",
"page": "11499",
"DOI": "10.1038/s41467-026-78777-2",
"PMID": "42995199",
"PMCID": "PMC13743752",
"ISSN": "2041-1723",
"publisher": "Nature Publishing Group",
"URL": "https://doi.org/10.1038/s41467-026-78777-2",
"language": "en",
"issued": {
"date-parts": [
[
2026,
11,
6
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1073/pnas.2408125122 [code]
A general framework for interpretable neural learning based on local information-theoretic goal functions.
Journal: Proceedings of the National Academy of Sciences of the United States of America
In common: seaborn, scikit-learn, SciPy, 2 other tools, 6 references, 5 authors
[2] doi:10.1371/journal.pcbi.1013500 [code]
Inferring effective networks of spiking neurons using a continuous-time estimator of transfer entropy.
Journal: PLoS computational biology
In common: PyTorch, seaborn, scikit-learn, 4 other tools, 1 reference, author Michael Wibral
[3] doi:10.1371/journal.pcbi.1011567 [code]
Information-theoretic analyses of neural data to minimize the effect of researchers' assumptions in predictive coding studies.
Journal: PLoS computational biology
In common: scikit-learn, pandas, SciPy, 2 other tools, 2 references, author Viola Priesemann
[4] doi:10.1038/s42003-025-07956-6 [code]
Power-law adaptation in the presynaptic vesicle cycle.
Journal: Communications biology
In common: h5py, seaborn, scikit-learn, 4 other tools, author Viola Priesemann
[5] doi:10.1038/s41593-023-01413-5 [code]
Propagation of activity through the cortical hierarchy and perception are determined by neural variability.
Journal: Nature neuroscience
In common: h5py, seaborn, scikit-learn, 4 other tools, author Viola Priesemann
[6] doi:10.1162/netn_a_00259 [code]
Conservative significance testing of tripartite statistical relations in multivariate neural data.
Journal: Network neuroscience (Cambridge, Mass.)
In common: h5py, seaborn, scikit-learn, 4 other tools, 3 references
[7] doi:10.1073/pnas.2500048122 [code]
Higher-order interactions in neuronal function: From genes to ionic currents in biophysical models.
Journal: Proceedings of the National Academy of Sciences of the United States of America
In common: seaborn, scikit-learn, pandas, 3 other tools, author Abdullah Makkeh
[8] doi:10.1371/journal.pcbi.1012355 [code]
Signatures of hierarchical temporal processing in the mouse visual system.
Journal: PLoS computational biology
In common: h5py, seaborn, pandas, 3 other tools, author Viola Priesemann
[9] doi:10.1126/sciadv.ade1755 [code]
Modular architecture facilitates noise-driven control of synchrony in neuronal networks.
Journal: Science advances
In common: h5py, seaborn, pandas, 3 other tools, author Viola Priesemann
[10] doi:10.1162/netn_a_00288 [code]
Significant subgraph mining for neural network inference with multiple comparisons correction.
Journal: Network neuroscience (Cambridge, Mass.)
In common: h5py, scikit-learn, SciPy, 2 other tools, author Michael Wibral

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2.svg)](https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2.svg
   :target: https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2"><img src="https://openscicode.org/badge/doi/10.1038/s41467-026-78777-2.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.