OSCR

Dendritic nonlinearities mitigate communication costs.

Code ↔ Paper

3 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 3 matches
  1. [1] § Methods › Datasets for machine learning experiments › Model architectures ↔ machine_learning_experiments/cifar/cifar/models/models_for_cifar/rir.py, lines 14 to 76 · score 0.77 · shortcut connection, convolutional layer, residual blocks, ReLU, ResNet, architecture · Report a mismatch
  2. [2] § Methods › Datasets for machine learning experiments › ImageNet dataset ↔ machine_learning_experiments/imagenet/imagenet/dataloader/cifar100.py, lines 10 to 54 · score 0.68 · random horizontal flipping, random cropping, jittering, ImageNet, training · Report a mismatch
  3. [3] § Methods › Datasets for machine learning experiments › ImageNet dataset ↔ machine_learning_experiments/cifar/cifar/dataloader/cifar10.py, lines 5 to 41 · score 0.67 · random horizontal flipping, random cropping, jittering, training · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 175 lines · 6.9 KB · CC-BY-4.0 · 1 match

  1. """resnet in resnet in pytorch
  5. [1] Sasha Targ, Diogo Almeida, Kevin Lyman.
  7. Resnet in Resnet: Generalizing Residual Architectures
  8. https://arxiv.org/abs/1603.08029v1
  9. """
  11. import torch
  12. import torch.nn as nn
  14. #geralized
  15. class ResnetInit(nn.Module):
  16. def __init__(self, in_channel, out_channel, stride):
  17. super().__init__()
  19. #"""The modular unit of the generalized residual network architecture is a
  20. #generalized residual block consisting of parallel states for a residual stream,
  21. #r, which contains identity shortcut connections and is similar to the structure
  22. #of a residual block from the original ResNet with a single convolutional layer
  23. #(parameters W l,r→r )
  24. self.residual_stream_conv = nn.Conv2d(in_channel, out_channel, 3, padding=1, stride=stride)
  26. #"""and a transient stream, t, which is a standard convolutional layer
  27. #(W l,t→t )."""
  28. self.transient_stream_conv = nn.Conv2d(in_channel, out_channel, 3, padding=1, stride=stride)
  30. #"""Two additional sets of convolutional filters in each block (W l,r→t , W l,t→r )
  31. #also transfer information across streams."""
  32. self.residual_stream_conv_across = nn.Conv2d(in_channel, out_channel, 3, padding=1, stride=stride)
  34. #"""We use equal numbers of filters for the residual and transient streams of the
  35. #generalized residual network, but optimizing this hyperparameter could lead to
  36. #further potential improvements."""
  37. self.transient_stream_conv_across = nn.Conv2d(in_channel, out_channel, 3, padding=1, stride=stride)
  39. self.residual_bn_relu = nn.Sequential(
  40. nn.BatchNorm2d(out_channel),
  41. nn.ReLU(inplace=True)
  42. )
  44. self.transient_bn_relu = nn.Sequential(
  45. nn.BatchNorm2d(out_channel),
  46. nn.ReLU(inplace=True)
  47. )
  49. #"""The form of the shortcut connection can be an identity function with
  50. #the appropriate padding or a projection as in He et al. (2015b)."""
  51. self.short_cut = nn.Sequential()
  52. if in_channel != out_channel or stride != 1:
  53. self.short_cut = nn.Sequential(
  54. nn.Conv2d(in_channel, out_channel, kernel_size=1, stride=stride)
  55. )
  58. def forward(self, x):
  59. x_residual, x_transient = x
  60. residual_r_r = self.residual_stream_conv(x_residual)
  61. residual_r_t = self.residual_stream_conv_across(x_residual)
  62. residual_shortcut = self.short_cut(x_residual)
  64. transient_t_t = self.transient_stream_conv(x_transient)
  65. transient_t_r = self.transient_stream_conv_across(x_transient)
  67. #transient_t_t = self.transient_stream_conv(x_residual)
  68. #transient_t_r = self.transient_stream_conv_across(x_residual)
  69. #"""Same-stream and cross-stream activations are summed (along with the
  70. #shortcut connection for the residual stream) before applying batch
  71. #normalization and ReLU nonlinearities (together σ) to get the output
  72. #states of the block (Equation 1) (Ioffe & Szegedy, 2015)."""
  73. x_residual = self.residual_bn_relu(residual_r_r + transient_t_r + residual_shortcut)
  74. x_transient = self.transient_bn_relu(residual_r_t + transient_t_t)
  76. return x_residual, x_transient
  80. class RiRBlock(nn.Module):
  81. def __init__(self, in_channel, out_channel, layer_num, stride, layer=ResnetInit):
  82. super().__init__()
  83. self.resnetinit = self._make_layers(in_channel, out_channel, layer_num, stride)
  85. #self.short_cut = nn.Sequential()
  86. #if stride != 1 or in_channel != out_channel:
  87. # self.short_cut = nn.Conv2d(in_channel, out_channel, kernel_size=1, stride=stride)
  89. def forward(self, x):
  90. x_residual, x_transient = self.resnetinit(x)
  91. #x_residual = x_residual + self.short_cut(x[0])
  92. #x_transient = x_transient + self.short_cut(x[1])
  94. return (x_residual, x_transient)
  96. #"""Replacing each of the convolutional layers within a residual
  97. #block from the original ResNet (Figure 1a) with a generalized residual block
  98. #(Figure 1b) leads us to a new architecture we call ResNet in ResNet (RiR)
  99. #(Figure 1d)."""
  100. def _make_layers(self, in_channel, out_channel, layer_num, stride, layer=ResnetInit):
  101. strides = [stride] + [1] * (layer_num - 1)
  102. layers = nn.Sequential()
  103. for index, s in enumerate(strides):
  104. layers.add_module("generalized layers{}".format(index), layer(in_channel, out_channel, s))
  105. in_channel = out_channel
  107. return layers
  109. class ResnetInResneet(nn.Module):
  110. def __init__(self, num_classes=100):
  111. super().__init__()
  112. base = int(96 / 2)
  113. self.residual_pre_conv = nn.Sequential(
  114. nn.Conv2d(3, base, 3, padding=1),
  115. nn.BatchNorm2d(base),
  116. nn.ReLU(inplace=True)
  117. )
  118. self.transient_pre_conv = nn.Sequential(
  119. nn.Conv2d(3, base, 3, padding=1),
  120. nn.BatchNorm2d(base),
  121. nn.ReLU(inplace=True)
  122. )
  124. self.rir1 = RiRBlock(base, base, 2, 1)
  125. self.rir2 = RiRBlock(base, base, 2, 1)
  126. self.rir3 = RiRBlock(base, base * 2, 2, 2)
  127. self.rir4 = RiRBlock(base * 2, base * 2, 2, 1)
  128. self.rir5 = RiRBlock(base * 2, base * 2, 2, 1)
  129. self.rir6 = RiRBlock(base * 2, base * 4, 2, 2)
  130. self.rir7 = RiRBlock(base * 4, base * 4, 2, 1)
  131. self.rir8 = RiRBlock(base * 4, base * 4, 2, 1)
  133. self.conv1 = nn.Sequential(
  134. nn.Conv2d(384, num_classes, kernel_size=3, stride=2), #without this convolution, loss will soon be nan
  135. nn.BatchNorm2d(num_classes),
  136. nn.ReLU(inplace=True),
  137. )
  139. self.classifier = nn.Sequential(
  140. nn.Linear(900, 450),
  141. nn.ReLU(),
  142. nn.Dropout(),
  143. nn.Linear(450, 100),
  144. )
  146. self._weight_init()
  148. def forward(self, x):
  149. x_residual = self.residual_pre_conv(x)
  150. x_transient = self.transient_pre_conv(x)
  152. x_residual, x_transient = self.rir1((x_residual, x_transient))
  153. x_residual, x_transient = self.rir2((x_residual, x_transient))
  154. x_residual, x_transient = self.rir3((x_residual, x_transient))
  155. x_residual, x_transient = self.rir4((x_residual, x_transient))
  156. x_residual, x_transient = self.rir5((x_residual, x_transient))
  157. x_residual, x_transient = self.rir6((x_residual, x_transient))
  158. x_residual, x_transient = self.rir7((x_residual, x_transient))
  159. x_residual, x_transient = self.rir8((x_residual, x_transient))
  160. h = torch.cat([x_residual, x_transient], 1)
  161. h = self.conv1(h)
  162. h = h.view(h.size()[0], -1)
  163. h = self.classifier(h)
  165. return h
  167. def _weight_init(self):
  168. for m in self.modules():
  169. if isinstance(m, nn.Conv2d):
  170. torch.nn.init.kaiming_normal(m.weight)
  171. m.bias.data.fill_(0.01)
  174. def resnet_in_resnet():
  175. return ResnetInResneet()

rir.py, under CC-BY-4.0 · at the source

Overview

Authors: Xundong Wu1,2,3, Pengfei Zhao2, Zilin Yu4, Lei Ma2,5, Yifan Gao1, Ka-Wa Yip1, Huajin Tang3, Gang Pan3, Panayiota Poirazi6, Tiejun Huang2,7
ORCID iDs: Xundong Wu
  1. Zhejiang Lab, Hangzhou, Zhejiang, China
  2. Beijing Academy of Artificial Intelligence, Beijing, China
  3. Zhejiang University, Hangzhou, China
  4. Bytedance, Beijing, China
  5. National Biomedical Imaging Center, Peking University, Beijing, China
  6. IMBB-FORTH, Heraklion, Crete, Greece
  7. National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China
Institutions: Beijing Academy of Artificial Intelligence (China); Zhejiang Lab (China); Zhejiang University (China); ByteDance (China) (China); Peking University (China); FORTH Institute of Molecular Biology and Biotechnology (Greece)
Journal: Patterns (New York, N.Y.), volume 7, issue 6, article 101520
Dates: received 6 November 2025; accepted 26 February 2026; published online 30 March 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1016/j.patter.2026.101520 · PMID 42328205 · PMCID PMC13280677 · OpenAlex W7143483948
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Methods: Preprocessing, Graphs, Machine learning
Keywords: dendritic nonlinearities, communication costs, localized feature aggregation, active dendrites, neural network, algorithm-hardware co-design, neuromorphic computing, brain-inspired computing, network scaling
Topic: Molecular Communication and Nanonetworks (Biomedical Engineering, Engineering), according to OpenAlex
Funding: NIMH NIH HHS (R01 MH124867)
Citations: not cited yet (Europe PMC); 44 references in the paper

Abstract

Why have modern artificial neural networks not adopted the nonlinear dendritic structures found in biological brain cells, and what is the core advantage of such active dendritic units? While early studies suggested that dendritic nonlinearities can enhance learning capabilities by boosting capacity, we provide empirical evidence reassessing this. Using extensive machine learning experiments, we show that dendritic nonlinearities in neural networks offer comparable learning capacity to standard point-neuron models when controlled for parametric complexity. Instead, we believe that their key advantage lies in enabling network scaling while substantially reducing communication costs via localized feature aggregation. Our experiments and analysis suggest that incorporating nonlinear dendritic architectures can significantly lower memory access or data transfer overhead during neural network inference—the primary sources of energy consumption in modern AI systems—and potentially during training as well. We argue that these insights motivate further theoretical and architectural exploration of dendritic-like structures in artificial neural networks.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 3 matches between paragraphs and lines of code.

motifMachine/Dendrite_communication

License: none: the authors keep all their rights
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: 475521f032ac732e9bac2aca6c5e3f4a31f5ee1f, 11 February 2026
Languages: Python (118), Shell (7)
Size: 133 files, 125 scripts
Software Heritage: not archived
Found in: “Data and code availability”
Holds: README
Not found: license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: PyTorch (100 files), NumPy (14 files)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
126 files
At the source: github.com/motifMachine/Dendrite_communication/

figshare 31305517

License: CC-BY-4.0
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Size: 1 file
Software Heritage: not checked
Found in: the references
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: PyTorch (100 files), NumPy (14 files)
Availability: 1 check, the latest on 30 September 2026: the link answers (HTTP 200)
  • 30 September 2026: the link answers (HTTP 200)
126 files
At the source:

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 250 scripts, each with its path and the digest of its content;
  • 3 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data and code availability

Our source code is available at GitHub (https://github.com/motifMachine/Dendrite_communication/) and has been archived at Figshare.44

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 30 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 10 authors, 9 keywords, 1 funder, 30 references.

Cite

This paper

Wu, X., Zhao, P., Yu, Z., Ma, L., Gao, Y., Yip, K.-W., Tang, H., Pan, G., Poirazi, P., & Huang, T. (2026). Dendritic nonlinearities mitigate communication costs. Patterns (New York, N.Y.), 7(6), 101520. https://doi.org/10.1016/j.patter.2026.101520

BibTeX

@article{wu2026dendritic,
author = {Wu, Xundong and Zhao, Pengfei and Yu, Zilin and Ma, Lei and Gao, Yifan and Yip, Ka-Wa and Tang, Huajin and Pan, Gang and Poirazi, Panayiota and Huang, Tiejun},
title = {{Dendritic nonlinearities mitigate communication costs}},
journal = {Patterns (New York, N.Y.)},
year = {2026},
month = mar,
volume = {7},
number = {6},
pages = {101520},
publisher = {Elsevier},
issn = {2666-3899},
doi = {10.1016/j.patter.2026.101520},
url = {https://doi.org/10.1016/j.patter.2026.101520},
pmid = {42328205},
pmcid = {PMC13280677}
}

RIS

TY - JOUR
AU - Wu, Xundong
AU - Zhao, Pengfei
AU - Yu, Zilin
AU - Ma, Lei
AU - Gao, Yifan
AU - Yip, Ka-Wa
AU - Tang, Huajin
AU - Pan, Gang
AU - Poirazi, Panayiota
AU - Huang, Tiejun
TI - Dendritic nonlinearities mitigate communication costs
T2 - Patterns (New York, N.Y.)
J2 - Patterns (N Y)
PY - 2026
DA - 2026/03/30
VL - 7
IS - 6
SP - 101520
SN - 2666-3899
PB - Elsevier
DO - 10.1016/j.patter.2026.101520
UR - https://doi.org/10.1016/j.patter.2026.101520
LA - en
ER -

CSL-JSON

{
"id": "10.1016/j.patter.2026.101520",
"type": "article-journal",
"title": "Dendritic nonlinearities mitigate communication costs",
"container-title": "Patterns (New York, N.Y.)",
"author": [
{
"family": "Wu",
"given": "Xundong"
},
{
"family": "Zhao",
"given": "Pengfei"
},
{
"family": "Yu",
"given": "Zilin"
},
{
"family": "Ma",
"given": "Lei"
},
{
"family": "Gao",
"given": "Yifan"
},
{
"family": "Yip",
"given": "Ka-Wa"
},
{
"family": "Tang",
"given": "Huajin"
},
{
"family": "Pan",
"given": "Gang"
},
{
"family": "Poirazi",
"given": "Panayiota"
},
{
"family": "Huang",
"given": "Tiejun"
}
],
"container-title-short": "Patterns (N Y)",
"volume": "7",
"issue": "6",
"page": "101520",
"DOI": "10.1016/j.patter.2026.101520",
"PMID": "42328205",
"PMCID": "PMC13280677",
"ISSN": "2666-3899",
"publisher": "Elsevier",
"URL": "https://doi.org/10.1016/j.patter.2026.101520",
"language": "en",
"issued": {
"date-parts": [
[
2026,
3,
30
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-025-56297-9 [code]
Dendrites endow artificial neural networks with accurate, robust and parameter-efficient learning.
Journal: Nature communications
In common: PyTorch, NumPy, 8 references
[2] doi:10.1016/j.celrep.2026.117159 [code]
Cellular and subcellular specialization enables biology-constrained deep learning.
Journal: Cell reports
In common: PyTorch, NumPy, 4 references
[3] doi:10.1371/journal.pbio.3002935 [code]
Local changes in potassium ions regulate input integration in active dendrites.
Journal: PLoS biology
In common: NumPy, 5 references
[4] doi:10.1016/j.isci.2025.112928 [code]
Of mice and men: Dendritic architecture differentiates human from mouse neuronal networks.
Journal: iScience
In common: NumPy, 5 references
[5] doi:10.1073/pnas.2533168123 [code]
Dendritic morphology and synaptic nonlinearities enhance functional complexity in human cortical neurons.
Journal: Proceedings of the National Academy of Sciences of the United States of America
In common: PyTorch, NumPy, 4 references
[6] doi:10.1038/s41467-023-41553-7 [code]
A GPU-based computational framework that bridges neuron simulation and artificial intelligence.
Journal: Nature communications
In common: NumPy, 4 references
[7] doi:10.1038/s41467-022-35747-8 [code]
Introducing the Dendrify framework for incorporating dendrites to spiking neural networks.
Journal: Nature communications
In common: NumPy, 4 references
[8] doi:10.1371/journal.pcbi.1012285 [code]
Parallel synapses with transmission nonlinearities enhance neuronal classification capacity.
Journal: PLoS computational biology
In common: PyTorch, NumPy, 3 references
[9] doi:10.7554/elife.89629 [code]
Active dendrites enable robust spiking computations despite timing jitter.
Journal: eLife
In common: NumPy, 4 references
[10] doi:10.1038/s42003-026-11157-0 [code]
A NEST-based framework unlocks massively parallel simulation of networks of multi-compartment neurons with customizable subcellular dynamics.
Journal: Communications biology
In common: NumPy, 3 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520.svg)](https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520.svg
   :target: https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520"><img src="https://openscicode.org/badge/doi/10.1016/j.patter.2026.101520.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.