Deep Learning for Content-Based Medical Image Retrieval in Picture Archiving and Communication Systems for Brain Tumor Detection: Algorithm Development and Validation.
Overview
- Department of Information Management, National Taipei University of Nursing and Health Science, No. 365, Ming-te Rd, Beitou Dist, Taipei City, 112303, Taiwan ext 1230
- Institute of Biophotonics, National Yang Ming Chiao Tung University, Taipei, Taiwan
- Department of Radiology, Taipei Veterans General Hospital, Taipei, Taiwan
- Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taipei, Taiwan
Abstract
Background: Advances in medical imaging have led to massive archives, yet navigating these datasets remains challenging due to the limitations of traditional text-based search engines. While content-based medical image retrieval (CBMIR) offers a visual feature–based solution to enhance clinical workflows and research, its operational integration into picture archiving and communication systems (PACS) remains a significant bottleneck. Despite the progress in deep learning for feature extraction, CBMIR tools are rarely integrated and effectively implemented within existing radiology information systems due to complex protocol barriers. Objective: To address these challenges, this study develops a CBMIR system meticulously designed to cater to 7 distinct types of brain tumors as seen in brain magnetic resonance images. Our system is tailored to assist radiologists and health care professionals in efficiently retrieving pertinent historical medical images, thereby providing quantitative decision support for radiologists and facilitating evidence-based case comparison, with the potential to improve retrieval efficiency and clinical workflow, rather than directly enhancing diagnostic accuracy. Methods: The dataset used in this study was collected from a single medical center and is not publicly available. The core innovation is a state-of-the-art deep learning–based feature extraction algorithm specifically engineered for the CBMIR system. We use GoogLeNet as the primary architecture, incorporating generalized mean pooling to capture nuanced local features and an embedding layer for dimension reduction. Crucially, we address the integration gap by harmonizing 2 open-source projects to successfully embed the CBMIR system into a functional PACS environment via standard protocols. Results: The image dataset contains 658 participants with 15,873 images collected from 2000 to 2017. The empirical findings of our research demonstrate the performance and robustness of the proposed CBMIR system. Our system achieves a remarkable mean average precision score of 89.16% and an equally impressive score of 94.08%. These metrics affirm the system’s efficacy in retrieving relevant medical images. Furthermore, we successfully integrate the CBMIR system into a PACS by successfully harmonizing 2 open-source projects. Conclusions: This study presents the design and implementation of a PACS-integrated CBMIR system for brain magnetic resonance imaging, and the experimental results demonstrate that the system can achieve efficient and accurate image retrieval within a clinical workflow
Reproduced under the paper's license (CC BY), from the paper cited above.
Code
The paper links to its data and to no code of its authors: see the Data section.
Tracing map
A tracing map links a paper to the code its authors published: this paper has none, so it has no map.
Data
Datasets cited
- figshare:1512427, at figshare; found in the references
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 1, 29 September 2026: the first record
Recorded: type, language, journal, volume, pages, dates, 6 authors, 9 keywords, 7 MeSH terms, 36 references.
Cite
This paper
Lee, C.-L., Hsu, T.-H., Wu, Y.-T., Guo, W.-Y., Chu, W.-C., & Lien, C.-Y. (2026). Deep Learning for Content-Based Medical Image Retrieval in Picture Archiving and Communication Systems for Brain Tumor Detection: Algorithm Development and Validation. JMIR medical informatics, 14, e78300. https://
BibTeX
@article{lee2026deep,
author = {Lee, Chin-Lin and Hsu, Tzu-Hsuan and Wu, Yu-Te and Guo, Wan-Yuo and Chu, Woei-Chyn and Lien, Chung-Yueh},
title = {{Deep Learning for Content-Based Medical Image Retrieval in Picture Archiving and Communication Systems for Brain Tumor Detection: Algorithm Development and Validation}},
journal = {JMIR medical informatics},
year = {2026},
month = apr,
volume = {14},
pages = {e78300},
publisher = {JMIR Publications Inc.},
issn = {2291-9694},
doi = {10.2196/
url = {https://
pmid = {41941563},
pmcid = {PMC13054792}
}
RIS
TY - JOUR
AU - Lee, Chin-Lin
AU - Hsu, Tzu-Hsuan
AU - Wu, Yu-Te
AU - Guo, Wan-Yuo
AU - Chu, Woei-Chyn
AU - Lien, Chung-Yueh
TI - Deep Learning for Content-Based Medical Image Retrieval in Picture Archiving and Communication Systems for Brain Tumor Detection: Algorithm Development and Validation
T2 - JMIR medical informatics
J2 - JMIR Med Inform
PY - 2026
DA - 2026/
VL - 14
SP - e78300
SN - 2291-9694
PB - JMIR Publications Inc.
DO - 10.2196/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.2196/
"type": "article-journal",
"title": "Deep Learning for Content-Based Medical Image Retrieval in Picture Archiving and Communication Systems for Brain Tumor Detection: Algorithm Development and Validation",
"container-title": "JMIR medical informatics",
"author": [
{
"family": "Lee",
"given": "Chin-Lin"
},
{
"family": "Hsu",
"given": "Tzu-Hsuan"
},
{
"family": "Wu",
"given": "Yu-Te"
},
{
"family": "Guo",
"given": "Wan-Yuo"
},
{
"family": "Chu",
"given": "Woei-Chyn"
},
{
"family": "Lien",
"given": "Chung-Yueh"
}
],
"container-title-short":
"volume": "14",
"page": "e78300",
"DOI": "10.2196/
"PMID": "41941563",
"PMCID": "PMC13054792",
"ISSN": "2291-9694",
"publisher": "JMIR Publications Inc.",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
4,
6
]
]
}
}
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1186/s12911-026-03367-7
- Dual-path deep learning framework for accurate and interpretable brain tumor diagnosis.Journal: BMC medical informatics and decision makingIn common: figshare 1512427, structural MRI / diffusion, other condition, 4 references
- [2] doi:10.3389/fninf.2025.1527582 [code]
- Contrastive self-supervised learning for neurodegenerative disorder classification.Journal: Frontiers in neuroinformaticsIn common: methods / tools, structural MRI / diffusion, other condition, 4 references
- [3] doi:10.3390/brainsci13091320
- Brain Tumor Classification from MRI Using Image Enhancement and Convolutional Neural Network Techniques.Journal: Brain sciencesIn common: figshare 1512427, structural MRI / diffusion, other condition, 2 references
- [4] doi:10.3389/fncom.2022.1004988
- GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID parallelism.Journal: Frontiers in computational neuroscienceIn common: figshare 1512427, other condition, 2 references
- [5] doi:10.1371/journal.pone.0284418
- Brain tumor detection and segmentation: Interactive framework with a visual interface and feedback facility for dynamically improved accuracy and trust.Journal: PloS oneIn common: figshare 1512427, methods / tools, structural MRI / diffusion, other condition, 1 reference
- [6] doi:10.3390/s22010372
- Brain Tumor/
Mass Classification Framework Using Magnetic-Resonance-Imagi ng-Based Isolated and Developed Transfer Deep-Learning Model. Journal: Sensors (Basel, Switzerland)In common: figshare 1512427, methods / tools, structural MRI / diffusion, other condition, 1 reference - [7] doi:10.3390/diagnostics16111745
- Brain Tumor Classification and Segmentation in MR Images Using EfficientNet and U-Net++ Models.Journal: Diagnostics (Basel, Switzerland)In common: figshare 1512427, methods / tools, structural MRI / diffusion, other condition, 1 reference
- [8] doi:10.3389/fmed.2025.1593016
- Optimal Res-UNET architecture with deep supervision for tumor segmentation.Journal: Frontiers in medicineIn common: figshare 1512427, methods / tools, structural MRI / diffusion, other condition, 1 reference
- [9] doi:10.3390/jimaging12060233 [code]
- Brain Tumor Classification in MRI Images Using Combined Transfer Learning and Convolutional Neural Networks.Journal: Journal of imagingIn common: figshare 1512427, structural MRI / diffusion, other condition, 1 reference
- [10] doi:10.3390/diagnostics14141469
- A Novel Hybrid Machine Learning-Based System Using Deep Learning Techniques and Meta-Heuristic Algorithms for Various Medical Datatypes Classification.Journal: Diagnostics (Basel, Switzerland)In common: figshare 1512427, methods / tools, other condition, 1 reference
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.
Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
