OSCR

Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber.

Code ↔ Paper

4 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 4 matches
  1. [1] § MATERIALS AND METHODS › Continuous super-resolution regression strategy ↔ 2-1-code.zip/model.py, lines 57–163 · score 0.77 · skip connections, ReLU, block, linear, channel, adaptively
  2. [2] § RESULTS › Continuous super-resolution tracking via geometric priors ↔ 2-2-code.zip/class_unet_fan_nosplit.py, lines 609–664 · score 0.67 · interval hit rate, adjacent anchors, unseen intervals, predicted angle
  3. [3] § MATERIALS AND METHODS › Discrete state confirmation ↔ 2-1-code.zip/model.py, lines 57–163 · score 0.62 · MaxPool2d, Conv2d, connected, blocks, activation, classifier
  4. [4] § MATERIALS AND METHODS › Discrete state confirmation ↔ 1-1-code.zip/code/class180.py, lines 88–127 · score 0.58 · MaxPool2d, Conv2d, Dropout, Tanh

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 163 lines · 6.4 KB · CC-BY-4.0 · 2 matches

  1. # # model.py
  3. import torch
  4. import torch.nn as nn
  5. import torch.nn.functional as F
  6. from typing import Optional
  9. # 定义通道注意力模块
  10. class ChannelAttentionModule(nn.Module):
  11. def __init__(self, channel: int, ratio: int = 16):
  12. super(ChannelAttentionModule, self).__init__()
  13. self.avg_pool = nn.AdaptiveAvgPool2d(1)
  14. self.max_pool = nn.AdaptiveMaxPool2d(1)
  16. # Shared 1×1 MLP implemented as Conv2d for efficiency (same weights for avg/max branches)
  17. self.shared_mlp = nn.Sequential(
  18. nn.Conv2d(channel, channel // ratio, kernel_size=1, bias=False),
  19. nn.ReLU(inplace=True),
  20. nn.Conv2d(channel // ratio, channel, kernel_size=1, bias=False)
  21. )
  22. self.sigmoid = nn.Sigmoid()
  24. def forward(self, x: torch.Tensor) -> torch.Tensor: # (B,C,H,W)
  25. avg_out = self.shared_mlp(self.avg_pool(x))
  26. max_out = self.shared_mlp(self.max_pool(x))
  27. return self.sigmoid(avg_out + max_out)
  30. # 定义空间注意力模块
  31. class SpatialAttentionModule(nn.Module):
  32. def __init__(self, kernel_size=7):
  33. super(SpatialAttentionModule, self).__init__()
  34. self.conv = nn.Conv2d(2, 1, kernel_size=kernel_size, padding=kernel_size//2)
  36. def forward(self, x):
  37. avg_out = torch.mean(x, dim=1, keepdim=True)
  38. max_out, _ = torch.max(x, dim=1, keepdim=True)
  39. x = torch.cat([avg_out, max_out], dim=1)
  40. x = self.conv(x)
  41. return torch.sigmoid(x)
  44. # 定义CBAM模块
  45. class CBAM(nn.Module):
  46. def __init__(self, channels, reduction=16, kernel_size=7):
  47. super(CBAM, self).__init__()
  48. self.channel_attention = ChannelAttentionModule(channels, reduction)
  49. self.spatial_attention = SpatialAttentionModule(kernel_size)
  51. def forward(self, x):
  52. x = x * self.channel_attention(x)
  53. x = x * self.spatial_attention(x)
  54. return x
  57. class UNet(nn.Module):
  58. def __init__(self, input_channels: int = 1, output_channels: int = 1, init_features: int = 64, num_classes: Optional[int] = None):
  59. super(UNet, self).__init__()
  60. features = init_features
  61. self.num_classes = num_classes
  63. # ---------------- Encoder ----------------
  64. self.encoder1 = self._block(input_channels, features)
  65. self.pool1 = nn.MaxPool2d(kernel_size=2, stride=2)
  66. self.encoder2 = self._block(features, features * 2)
  67. self.pool2 = nn.MaxPool2d(kernel_size=2, stride=2)
  68. self.encoder3 = self._block(features * 2, features * 4)
  69. self.pool3 = nn.MaxPool2d(kernel_size=2, stride=2)
  70. self.encoder4 = self._block(features * 4, features * 8)
  71. self.pool4 = nn.MaxPool2d(kernel_size=2, stride=2)
  73. # ---------------- Bottleneck ----------------
  74. self.bottleneck = self._block(features * 8, features * 16)
  76. # ---------------- Decoder ----------------
  77. self.upconv4 = nn.ConvTranspose2d(features * 16, features * 8, kernel_size=2, stride=2)
  78. self.decoder4 = self._block(features * 16, features * 8)
  79. self.upconv3 = nn.ConvTranspose2d(features * 8, features * 4, kernel_size=2, stride=2)
  80. self.decoder3 = self._block(features * 8, features * 4)
  81. self.upconv2 = nn.ConvTranspose2d(features * 4, features * 2, kernel_size=2, stride=2)
  82. self.decoder2 = self._block(features * 4, features * 2)
  83. self.upconv1 = nn.ConvTranspose2d(features * 2, features, kernel_size=2, stride=2)
  84. self.decoder1 = self._block(features * 2, features)
  86. # ---------------- Output ----------------
  87. self.conv_final = nn.Conv2d(features, output_channels, kernel_size=1)
  88. self.activation = nn.Sigmoid()
  90. # ---------------- CBAM Attention ----------------
  91. self.cbam1 = CBAM(features)
  92. self.cbam2 = CBAM(features * 2)
  93. self.cbam3 = CBAM(features * 4)
  94. self.cbam4 = CBAM(features * 8)
  96. # ----------- 分类头（阶段1用） -----------
  97. if num_classes is not None:
  98. self.gap = nn.AdaptiveAvgPool2d(1)
  99. self.flatten = nn.Flatten()
  100. self.classifier = nn.Linear(features * 16, num_classes)
  102. def _block(self, in_channels, out_channels):
  103. return nn.Sequential(
  104. nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1),
  105. nn.BatchNorm2d(out_channels),
  106. nn.ReLU(inplace=True),
  107. nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1),
  108. nn.BatchNorm2d(out_channels),
  109. nn.ReLU(inplace=True)
  110. )
  112. def _encode(self, x: torch.Tensor) -> torch.Tensor:
  113. enc1 = self.encoder1(x)
  114. enc1 = self.cbam1(enc1) + enc1
  115. enc2 = self.encoder2(self.pool1(enc1))
  116. enc2 = self.cbam2(enc2) + enc2
  117. enc3 = self.encoder3(self.pool2(enc2))
  118. enc3 = self.cbam3(enc3) + enc3
  119. enc4 = self.encoder4(self.pool3(enc3))
  120. enc4 = self.cbam4(enc4) + enc4
  121. bottleneck = self.bottleneck(self.pool4(enc4))
  122. return bottleneck
  124. def forward_classify(self, x: torch.Tensor) -> torch.Tensor:
  125. feats = self._encode(x)
  126. pooled = self.gap(feats)
  127. flat = self.flatten(pooled)
  128. logits = self.classifier(flat)
  129. return logits
  131. def forward(self, x: torch.Tensor) -> torch.Tensor:
  132. # -------- 编码器 --------
  133. bottleneck = self._encode(x)
  135. # -------- 解码器 --------
  136. # 需要重新获取enc1-enc4用于skip connection
  137. enc1 = self.encoder1(x)
  138. enc1 = self.cbam1(enc1) + enc1
  139. enc2 = self.encoder2(self.pool1(enc1))
  140. enc2 = self.cbam2(enc2) + enc2
  141. enc3 = self.encoder3(self.pool2(enc2))
  142. enc3 = self.cbam3(enc3) + enc3
  143. enc4 = self.encoder4(self.pool3(enc3))
  144. enc4 = self.cbam4(enc4) + enc4
  146. dec4 = self.upconv4(bottleneck)
  147. dec4 = torch.cat((dec4, enc4), dim=1)
  148. dec4 = self.decoder4(dec4)
  150. dec3 = self.upconv3(dec4)
  151. dec3 = torch.cat((dec3, enc3), dim=1)
  152. dec3 = self.decoder3(dec3)
  154. dec2 = self.upconv2(dec3)
  155. dec2 = torch.cat((dec2, enc2), dim=1)
  156. dec2 = self.decoder2(dec2)
  158. dec1 = self.upconv1(dec2)
  159. dec1 = torch.cat((dec1, enc1), dim=1)
  160. dec1 = self.decoder1(dec1)
  162. out = self.conv_final(dec1)
  163. return self.activation(out)

model.py, under CC-BY-4.0 · at the source

Overview

Authors: Zhaofan He1,2, Lele Wang1,2, Haidi Geng1,2, Zhengyang Lu3, Tiantian He1,2, Hongkun Zhong1,2, Hailong Zhang1,2, Runfeng Zhu4, Qingxiang Zhao4, Yuan Meng1, Dan Li1,2, Ping Yan1,2, Qiang Liu1,2, Qirong Xiao1,2
ORCID iDs: Zhaofan He, Lele Wang, Tiantian He, Hailong Zhang, Runfeng Zhu, Qingxiang Zhao, Yuan Meng, Qiang Liu, Qirong Xiao
  1. Department of Precision Instrument, Tsinghua University, Beijing 100084, China
  2. State Key Laboratory of Precision Space-time Information Sensing Technology, Beijing 100084, China
  3. Weixian College, Tsinghua University, Beijing 100084, China
  4. West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610041, China
Institutions: Tsinghua University (China); Sichuan University (China); West China Medical Center of Sichuan University (China); West China Hospital of Sichuan University (China)
Journal: Science advances, volume 12, issue 24, article eaef6263
Dates: received 20 January 2026; accepted 30 April 2026; published online 12 June 2026; in print June 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1126/sciadv.aef6263 · PMID 42284416 · PMCID PMC13262633 · OpenAlex W7164568522
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: cognitive (subfield)
Methods: Connectivity, Smoothing, state filtering, decompositions, Machine learning, fMRI & imaging
Journal subjects: Physical and Materials Sciences, Optics, Machine Learning
Topic: Soft Robotics and Applications (Biomedical Engineering, Engineering), according to OpenAlex
Funding: National Natural Science Foundation of China (National Science Foundation of China) (62475132); Beijing Natural Science Foundation (L241021)
Citations: cited by 1 paper (Europe PMC); 53 references in the paper

Abstract

The evolution of soft robots into embodied intelligent systems relies fundamentally on precise proprioception. However, a universal solution for capturing continuous deformations during diverse interactions, particularly in spatially confined interventional scenarios, remains lacking. Here, we introduce a deep learning–enabled versatile shape perception method based on a single-ended multimode fiber (MMF). By leveraging the intrinsic integration advantages of optics, our minimalist reflective architecture physically eliminates the dependence on complex demodulation units and distal devices. Furthermore, treating chaotic optical speckle fields as data streams encoding high-dimensional shape information, reconfigurable neural decoders resolve a single physical channel into versatile perception modes tailored to heterogeneous tasks: discrete state confirmation on soft grippers (>99% accuracy), continuous shape tracking on bionic dexterous hands (~5-fold spatial resolution enhancement), and intuitive 3D morphological reconstruction of soft surgical robots (IoU>0.93). Overall, our work establishes a versatile framework for breaking hardware adaptability limits via computation, laying a solid foundation for closed-loop control in digital twins of soft robots.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.

Zenodo 19601658

License: CC-BY-4.0
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Size: 18 files
Software Heritage: not checked
Found in: “Data, code, and materials availability:”
Holds: README
Not found: license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (23 files), PyTorch (16 files), Matplotlib (12 files), pandas (9 files), Pillow (5 files), scikit-learn (5 files), scikit-image (4 files), OpenCV (3 files), SciPy (3 files), seaborn (3 files), NetworkX (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
  • 27 September 2026: the link answers (HTTP 200)
28 files
At the source: zenodo.org/records/19601658

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 27 scripts, each with its path and the digest of its content;
  • 4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data, code, and materials availability

All data and code needed to evaluate and reproduce the results in the paper are present in the paper and/or the Supplementary Materials. This study did not generate new materials. All experimental data, source code, and supplementary results supporting the findings of this study are publicly available at Zenodo (https://zenodo.org/records/19601658).

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 14 authors, 2 funders, 33 references.

Cite

This paper

He, Z., Wang, L., Geng, H., Lu, Z., He, T., Zhong, H., Zhang, H., Zhu, R., Zhao, Q., Meng, Y., Li, D., Yan, P., Liu, Q., & Xiao, Q. (2026). Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber. Science advances, 12(24), eaef6263. https://doi.org/10.1126/sciadv.aef6263

BibTeX

@article{he2026deep,
author = {He, Zhaofan and Wang, Lele and Geng, Haidi and Lu, Zhengyang and He, Tiantian and Zhong, Hongkun and Zhang, Hailong and Zhu, Runfeng and Zhao, Qingxiang and Meng, Yuan and Li, Dan and Yan, Ping and Liu, Qiang and Xiao, Qirong},
title = {{Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber}},
journal = {Science advances},
year = {2026},
month = jun,
volume = {12},
number = {24},
pages = {eaef6263},
publisher = {American Association for the Advancement of Science},
issn = {2375-2548},
doi = {10.1126/sciadv.aef6263},
url = {https://doi.org/10.1126/sciadv.aef6263},
pmid = {42284416},
pmcid = {PMC13262633}
}

RIS

TY - JOUR
AU - He, Zhaofan
AU - Wang, Lele
AU - Geng, Haidi
AU - Lu, Zhengyang
AU - He, Tiantian
AU - Zhong, Hongkun
AU - Zhang, Hailong
AU - Zhu, Runfeng
AU - Zhao, Qingxiang
AU - Meng, Yuan
AU - Li, Dan
AU - Yan, Ping
AU - Liu, Qiang
AU - Xiao, Qirong
TI - Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber
T2 - Science advances
J2 - Sci Adv
PY - 2026
DA - 2026/06/12
VL - 12
IS - 24
SP - eaef6263
SN - 2375-2548
PB - American Association for the Advancement of Science
DO - 10.1126/sciadv.aef6263
UR - https://doi.org/10.1126/sciadv.aef6263
LA - en
ER -

CSL-JSON

{
"id": "10.1126/sciadv.aef6263",
"type": "article-journal",
"title": "Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber",
"container-title": "Science advances",
"author": [
{
"family": "He",
"given": "Zhaofan"
},
{
"family": "Wang",
"given": "Lele"
},
{
"family": "Geng",
"given": "Haidi"
},
{
"family": "Lu",
"given": "Zhengyang"
},
{
"family": "He",
"given": "Tiantian"
},
{
"family": "Zhong",
"given": "Hongkun"
},
{
"family": "Zhang",
"given": "Hailong"
},
{
"family": "Zhu",
"given": "Runfeng"
},
{
"family": "Zhao",
"given": "Qingxiang"
},
{
"family": "Meng",
"given": "Yuan"
},
{
"family": "Li",
"given": "Dan"
},
{
"family": "Yan",
"given": "Ping"
},
{
"family": "Liu",
"given": "Qiang"
},
{
"family": "Xiao",
"given": "Qirong"
}
],
"container-title-short": "Sci Adv",
"volume": "12",
"issue": "24",
"page": "eaef6263",
"DOI": "10.1126/sciadv.aef6263",
"PMID": "42284416",
"PMCID": "PMC13262633",
"ISSN": "2375-2548",
"publisher": "American Association for the Advancement of Science",
"URL": "https://doi.org/10.1126/sciadv.aef6263",
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
12
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-74358-5 [code]
Brain-inspired spatial intelligence for embodied agents.
Journal: Nature communications
In common: NetworkX, OpenCV, Pillow, 7 other tools, cognitive, 1 reference
[2] doi:10.1038/s42003-026-10957-8 [code]
Brain defence by the extracellular matrix protein Cochlin.
Journal: Communications biology
In common: NetworkX, OpenCV, scikit-image, 8 other tools
[3] doi:10.1371/journal.pcbi.1014571 [code]
SynAPSeg: A novel dataset and image analysis framework for deep learning-based synapse detection and quantification.
Journal: PLoS computational biology
In common: NetworkX, OpenCV, scikit-image, 8 other tools
[4] doi:10.1016/j.isci.2026.116825 [code]
Social hierarchy shapes behavioral and transcriptional responses to chronic stress and ketamine in male mice.
Journal: iScience
In common: NetworkX, OpenCV, scikit-image, 8 other tools
[5] doi:10.1016/j.isci.2026.116206 [code]
Gut distension evokes rapid neural dynamics in vagal and hindbrain populations of larval zebrafish.
Journal: iScience
In common: NetworkX, OpenCV, scikit-image, 8 other tools
[6] doi:10.1038/s41593-026-02232-0 [code]
Entorhinal cortex represents task-relevant remote locations independently of CA1.
Journal: Nature neuroscience
In common: NetworkX, OpenCV, scikit-image, 8 other tools
[7] doi:10.1038/s41598-026-43529-1 [code]
A spiking neural network inspired by neuroscience and psychology for Western mode- and key-conditioned music learning and composition.
Journal: Scientific reports
In common: NetworkX, OpenCV, Pillow, 7 other tools, cognitive
[8] doi:10.1126/sciadv.aed3650 [code]
Truthful visualizations for mass spectrometry imaging enable high-spatial-resolution interactive <i>m/z</i> mapping and exploration.
Journal: Science advances
In common: NetworkX, OpenCV, scikit-image, 7 other tools
[9] doi:10.7554/elife.110074 [code]
Disentangling cephalopod chromatophores motor units with computer vision.
Journal: eLife
In common: NetworkX, OpenCV, scikit-image, 7 other tools
[10] doi:10.1038/s41598-026-57519-w [code]
Automated segmentation of neurons and spinal cord structures in immunofluorescence images using SpineDL.
Journal: Scientific reports
In common: NetworkX, OpenCV, scikit-image, 7 other tools

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.