Visual prompt engineering for multimodal and irregularly sampled medical data.
5 blocks of code here have a permanent identifier (oscr:b:…), listed with the matches below: citable for good, with a page of its own. Select a match and press Cite to copy it. What a block identifier is.
The 7 matches
- [1] § Results › Interpretability with attention visualization ↔ patient_journey.ipynb, lines 93 to 209 · score 0.86 · Invasive Blood Pressure, Respiratory Rate, Heart Rate, ICU patients, vital signs, Diastolic · Report a mismatch
- [2] § Methods › Data and benchmarks › MIMIC-IV ↔ data.py, lines 15 to 78 · score 0.59 · acute myocardial infarction, chronic, shock, Phenotype · block
oscr:b:3C8P1HB5K4F1N-5· Report a mismatch - [3] § Methods › Data and benchmarks › MIMIC-IV ↔ data2.py, lines 1 to 36 · score 0.59 · acute myocardial infarction, chronic, shock, Phenotype · block
oscr:b:N619YRDDZMMT2-W· Report a mismatch - [4] § Methods › Architecture with vision transformers ↔ model.py, lines 6 to 87 · score 0.56 · text model, linear, RoBERTa, pretrained, classification, Swin · block
oscr:b:JADDZ7B5WBVE6-V· Report a mismatch - [5] § Methods › Data and benchmarks › CDSL ↔ main.py, lines 201 to 231 · score 0.54 · HM Hospitales, hospital mortality, multimodal, COVID, CDSL, PhysioNet · block
oscr:b:TG71A7FY80PP7-K· Report a mismatch - [6] § Methods › Data and benchmarks › CDSL ↔ data2.py, lines 174 to 281 · score 0.51 · HM Hospitales, hospital mortality, multimodal, COVID, CDSL, diagnoses · block
oscr:b:AXZXR0GY4Z6RD-J· Report a mismatch - [7] § Results › Uni-modal training › MIMIC In-hospital mortality ↔ eval_inhospital_mortality.ipynb, lines 164 to 182 · score 0.51 · Bal.Acc, balanced accuracy, AUPRC, mortality · Report a mismatch
Paper
Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
Python · 348 lines · 12 KB · no license · 2 matches
- from pathlib import Path
- from typing import List
- from PIL import Image
- import pandas as pd
- import torch
- from torch.utils.data import DataLoader
- from torchvision import transforms
- from transformers import ViTFeatureExtractor, AutoTokenizer
- PHENOTYPING_COLUMNS = [
- 'Acute and unspecified renal failure',
- 'Acute cerebrovascular disease',
- 'Acute myocardial infarction',
- 'Cardiac dysrhythmias',
- 'Chronic kidney disease',
- 'Chronic obstructive pulmonary disease and bronchiectasis',
- 'Complications of surgical procedures or medical care',
- 'Conduction disorders',
- 'Congestive heart failure; nonhypertensive',
- 'Coronary atherosclerosis and other heart disease',
- 'Diabetes mellitus with complications',
- 'Diabetes mellitus without complication',
- 'Disorders of lipid metabolism',
- 'Essential hypertension',
- 'Fluid and electrolyte disorders',
- 'Gastrointestinal hemorrhage',
- 'Hypertension with complications and secondary hypertension',
- 'Other liver diseases',
- 'Other lower respiratory disease',
- 'Other upper respiratory disease',
- 'Pleurisy; pneumothorax; pulmonary collapse',
- 'Pneumonia (except that caused by tuberculosis or sexually transmitted disease)',
- 'Respiratory failure; insufficiency; arrest (adult)',
- 'Septicemia (except in labor)',
- 'Shock'
- ]
- def number_to_bucket(value):
- """Convert continuous number to bucket classification"""
- # if pd.isna(value):
- # return np.nan
- if 0 <= value < 1:
- return 0
- elif 1 <= value < 2:
- return 1
- elif 2 <= value < 3:
- return 2
- elif 3 <= value < 4:
- return 3
- elif 4 <= value < 5:
- return 4
- elif 5 <= value < 6:
- return 5
- elif 6 <= value < 7:
- return 6
- elif 7 <= value < 8:
- return 7
- elif 8 <= value <= 14:
- return 8
- else: # > 14
- return 9
- class MIMICDataset:
- def __init__(
- self,
- task,
- split,
- modalities,
- with_diagnoses,
- root='/mnt/hdd/data/ViTiMM_data2/data/',
- cxr_root='/mnt/sds/sd20i001/malte/data/physionet.org/files/mimic-cxr-jpg',
- non_empty=[],
- timeframe=48,
- image_size=384,
- max_length=128,
- img_model_name='microsoft/swin-large-patch4-window12-384-in22k',
- *args, **kwargs
- ):
- self.root = Path(root)
- self.src = self.root / 'plots' / task / split if task in ['in-hospital-mortality','phenotyping'] else self.root / task / str(timeframe) / split
- self.cxr_root = Path(cxr_root)
- self.task = task
- self.split = split
- self.modalities = modalities
- self.with_diagnoses = with_diagnoses
- self.df = pd.read_csv(self.root / f'{task}_{split}.csv', index_col=0)
- # if non_empty:
- self.non_empty = non_empty
- if len(non_empty):
- self.df = self.df.dropna(subset=[f'{m}_fname' for m in non_empty], how='any')
- self.tokenizer_img = ViTFeatureExtractor.from_pretrained(img_model_name) # 'google/vit-base-patch16-224')
- self.cxr_train_transform = transforms.Compose([
- transforms.RandomHorizontalFlip(),
- transforms.RandomAffine(degrees=45, scale=(.85, 1.15), shear=0, translate=(0.15, 0.15))
- ])
- self.image_size = image_size
- self.img_transform = transforms.Compose([
- transforms.Resize((image_size, image_size)),
- transforms.ToTensor(),
- transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
- ])
- self.text_tokenizer = AutoTokenizer.from_pretrained('roberta-base')
- self.max_length = max_length
- # self.empty_ecg_path = self.root / 'ecgs2' / 'empty_ecg.png'
- def __getitem__(self, i):
- sample = self.df.iloc[i]
- if self.task == 'phenotyping':
- label = sample[PHENOTYPING_COLUMNS].astype(float).tolist()
- elif self.task == 'length-of-stay':
- label = sample.y_true
- label = number_to_bucket(label)
- elif self.task == 'decompensation':
- label = [sample.y_true]
- else: # inhospital_mortality
- label = [sample.y_true]
- get_text = lambda k: sample[f'{k}_text'] if type(sample[f'{k}_text']) == str else ''
- text = get_text('meta')
- if self.with_diagnoses:
- text += ' ' + get_text('diagnoses')
- if type(sample.ecg_text) == str:
- text += ' ' + get_text('ecg')
- if type(sample.cxr_text) == str:
- text += ' ' + get_text('cxr')
- if type(sample.med_text) == str:
- text += ' ' + get_text('med')
- data = {'y': torch.tensor(label, dtype=torch.float32)}
- if 'lab' in self.modalities:
- data['lab'] = self.load_image(sample.lab_fname)
- if 'med' in self.modalities:
- data['med'] = self.load_image(sample.med_fname)
- if 'cxr' in self.modalities:
- data['cxr'] = self.load_image(sample.cxr_fname,'cxr')
- if 'ecg' in self.modalities:
- data['ecg'] = self.load_image(sample.ecg_fname)
- text_inputs = self.text_tokenizer(
- text,
- padding='max_length',
- max_length=self.max_length,
- return_tensors='pt',
- truncation=True
- )
- data['input_ids'] = text_inputs['input_ids'].squeeze()
- data['attention_mask'] = text_inputs['attention_mask'].squeeze()
- return data
- def load_image(self, fname, modality=None):
- if type(fname) == str:
- if not fname.endswith('.png') and not fname.endswith('.jpg'):
- fname += '.png'
- fname2 = self.root / fname if modality != 'cxr' else self.cxr_root / fname
- img = Image.open(fname2).convert('RGB')
- img = self.img_transform(img)
- else:
- img = torch.zeros((3,self.image_size, self.image_size)).float()
- return img
- def __len__(self):
- return len(self.df)
- class CovidDataset:
- def __init__(
- self,
- task,
- split,
- modalities,
- with_diagnoses,
- root='/mnt/hdd/data/covid-data-for-shared-learning-cdsl-a-comprehensive-multimodal-covid-19-dataset-from-hm-hospitales-1.0.0/',
- non_empty=[],
- image_size=384,
- max_length=128,
- img_model_name='microsoft/swin-large-patch4-window12-384-in22k',
- *args, **kwargs
- ):
- self.root = Path(root)
- self.src = self.root / 'plots' #task / split if task in ['in-hospital-mortality','phenotyping'] else self.root / task / str(timeframe) / split
- self.task = task
- self.split = split
- self.modalities = modalities
- self.with_diagnoses = with_diagnoses
- self.df = pd.read_csv(self.root / 'meta.csv', index_col=0)
- self.df = self.df.dropna(subset=['inhospital_mortality','length_of_stay'], how='any')
- self.df = self.df[self.df.split==split]
- self.df = self.df.dropna(subset=['med_fname','lab_fname','cxr_fname'], how='all')
- self.df = self.df[self.df.length_of_stay>=5]
- self.non_empty = non_empty
- if len(non_empty):
- self.df = self.df.dropna(subset=[f'{m}_fname' for m in non_empty], how='any')
- self.tokenizer_img = ViTFeatureExtractor.from_pretrained(img_model_name) # 'google/vit-base-patch16-224')
- self.cxr_train_transform = transforms.Compose([
- transforms.RandomHorizontalFlip(),
- transforms.RandomAffine(degrees=45, scale=(.85, 1.15), shear=0, translate=(0.15, 0.15))
- ])
- self.image_size = image_size
- self.img_transform = transforms.Compose([
- transforms.Resize((image_size, image_size)),
- transforms.ToTensor(),
- transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
- ])
- self.text_tokenizer = AutoTokenizer.from_pretrained('roberta-base')
- self.max_length = max_length
- # self.empty_ecg_path = self.root / 'ecgs2' / 'empty_ecg.png'
- def __getitem__(self, i):
- sample = self.df.iloc[i]
- # if self.task == 'phenotyping':
- # label = sample[PHENOTYPING_COLUMNS].astype(float).tolist()
- if self.task == 'length-of-stay':
- label = sample.length_of_stay
- label = number_to_bucket(label) - 5
- else: # inhospital_mortality
- label = [sample.inhospital_mortality]
- get_text = lambda k: sample[f'{k}_text'] if type(sample[f'{k}_text']) == str else ''
- text = get_text('meta')
- if self.with_diagnoses:
- text += ' ' + get_text('diag')
- # if type(sample.ecg_text) == str:
- # text += ' ' + get_text('ecg')
- # if type(sample.cxr_text) == str:
- # text += ' ' + get_text('cxr')
- if type(sample.med_text) == str:
- text += ' ' + get_text('med')
- data = {'y': torch.tensor(label, dtype=torch.float32)}
- if 'lab' in self.modalities:
- data['lab'] = self.load_image(sample.lab_fname)
- if 'med' in self.modalities:
- data['med'] = self.load_image(sample.med_fname)
- if 'cxr' in self.modalities:
- data['cxr'] = self.load_image(sample.cxr_fname)
- # if 'ecg' in self.modalities:
- # data['ecg'] = self.load_image(sample.ecg_fname)
- text_inputs = self.text_tokenizer(
- text,
- padding='max_length',
- max_length=self.max_length,
- return_tensors='pt',
- truncation=True
- )
- data['input_ids'] = text_inputs['input_ids'].squeeze()
- data['attention_mask'] = text_inputs['attention_mask'].squeeze()
- return data
- def load_image(self, fname):
- if type(fname) == str:
- if not fname.endswith('.png') and not fname.endswith('.jpg'):
- fname += '.png'
- fname = self.src / fname
- img = Image.open(fname).convert('RGB')
- img = self.img_transform(img)
- else:
- img = torch.zeros((3,self.image_size, self.image_size)).float()
- return img
- def __len__(self):
- return len(self.df)
- def dataloaders(
- dataset: str,
- task: str,
- modalities: List[str],
- non_empty: bool = False,
- image_size: int = 224,
- img_model_name: str = 'microsoft/swin-base-patch4-window7-224-in22k',
- batch_size: int = 16,
- root: str = './data',
- cxr_root: str = './cxrs',
- with_diagnoses: bool = True
- ):
- if dataset == 'covid' and task in ['decompensation','phenotyping']:
- raise ValueError
- dataset = MIMICDataset if dataset == 'mimic' else CovidDataset
- trainset = dataset(
- task=task,
- split='train',
- modalities=modalities,
- non_empty=non_empty,
- with_diagnoses=with_diagnoses,
- root=root,
- cxr_root=cxr_root,
- timeframe=48,
- max_length=512,
- image_size=image_size,
- img_model_name=img_model_name
- )
- from sklearn.model_selection import train_test_split
- train_df = trainset.df
- train_df, val_df = train_test_split(train_df, test_size=0.1, random_state=42)
- valset = dataset(
- task=task,
- split='train',
- modalities=modalities,
- non_empty=non_empty,
- with_diagnoses=with_diagnoses,
- root=root,
- cxr_root=cxr_root,
- timeframe=48,
- max_length=512,
- image_size=image_size,
- img_model_name=img_model_name
- )
- valset.df = val_df
- trainset.df = train_df
- testset = dataset(
- task=task,
- split='test',
- modalities=modalities,
- non_empty=non_empty,
- with_diagnoses=with_diagnoses,
- root=root,
- cxr_root=cxr_root,
- timeframe=48,
- max_length=512,
- image_size=image_size,
- img_model_name=img_model_name
- )
- trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True)#, num_workers=0, worker_init_fn=worker_init_fn, generator=g)
- valloader = DataLoader(valset, batch_size=batch_size)#, num_workers=0, worker_init_fn=worker_init_fn, generator=g)
- testloader = DataLoader(testset, batch_size=batch_size)#, num_workers=0, worker_init_fn=worker_init_fn, generator=g)
- return trainloader, valloader, testloader
data2.py at commit 4ae9b54, no license · at the source
Overview
- Department of Cardiology, Angiology and Pneumology, Heidelberg University Hospital, Heidelberg, Germany
- Informatics for Life Institute, Heidelberg, Germany
- DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim, Heidelberg, Germany
- Preventive Cardiology and Preventive Medicine, Department of Cardiology, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany
- Clinical Epidemiology and Systems Medicine, Center for Thrombosis and Hemostasis, University Medical Center Mainz, Johannes Gutenberg University Mainz, Mainz, Germany
- DZHK (German Centre for Cardiovascular Research), Partner Site Rhine-Main, Mainz, Germany
- Systems Medicine, Institute of Molecular Biology (IMB), Mainz, Germany
Abstract
Background: A patient undergoes multiple examinations in each hospital stay, where each provides different facets of the health status. These assessments include temporal data with varying sampling rates, discrete single-point measurements, therapeutic interventions such as medication administration, and images. While physicians are able to process and integrate diverse modalities intuitively, neural networks need specific modeling for each modality complicating the training procedure.
Methods: We demonstrate that this complexity can be significantly reduced by visualizing all information as images along with unstructured text and subsequently training a conventional vision-text transformer. Our approach, Vision Transformer for irregular sampled Multi-modal Measurements (ViTiMM), simplifies data preprocessing and modeling by unifying clinical measurements, medications, X-ray images, and electrocardiography scans into a single visual representation.
Results: ViTiMM outperforms current state-of-the-art methods in predicting in-hospital mortality, phenotyping, and decompensation on two datasets, the MIMIC-IV and COVID Data for Shared Learning (CDSL) dataset.
Conclusions: We hope our work inspires advancements in multi-modal medical AI by reducing the training complexity to (visual) prompt engineering, thus lowering entry barriers and enabling no-code solutions for training. The source code is publicly available at https://
Reproduced under the paper's license (CC BY), from the paper cited above.
Repositories
Its files are read in the Code ↔ Paper reader above, with 7 matches between paragraphs and lines of code.
Zenodo 20786553
Availability: 1 check, the latest on 26 September 2026: the link answers (HTTP 200)
- 26 September 2026: the link answers (HTTP 200)
17 files
- attention.ipynb, Jupyter, 129 lines
- data.py, Python, 228 lines
- data2.py, Python, 348 lines
- eval_dataloader.ipynb, Jupyter, 28 lines
- eval_inhospital_mortalit
y.ipynb , Jupyter, 188 lines - eval_phenotyping.ipynb, Jupyter, 181 lines
- main.py, Python, 231 lines
- main_mimic.py, Python, 215 lines
- model.py, Python, 137 lines
- p_values_inhospital_mort
ality.ipynb , Jupyter, 106 lines - p_values_phenotyping.ipy
nb , Jupyter, 104 lines - patient_journey.ipynb, Jupyter, 215 lines
- plot_ecgs.ipynb, Jupyter, 74 lines
- plot_labs.ipynb, Jupyter, 109 lines
- plot_meds.ipynb, Jupyter, 312 lines
- utils.py, Python, 200 lines
- README.md, Text, 178 lines
Cardio-AI/ViTiMM
4ae9b544665e1a8b5be2ee81b1622c21ca4fe115, 5 August 2025Availability: 1 check, the latest on 26 September 2026: the link answers
- 26 September 2026: the link answers
17 files
- attention.ipynb, Jupyter, 129 lines
- data.py, Python, 228 lines, 1 match
- data2.py, Python, 348 lines, 2 matches
- eval_dataloader.ipynb, Jupyter, 28 lines
- eval_inhospital_mortalit
y.ipynb , Jupyter, 188 lines, 1 match - eval_phenotyping.ipynb, Jupyter, 181 lines
- main.py, Python, 231 lines, 1 match
- main_mimic.py, Python, 215 lines
- model.py, Python, 137 lines, 1 match
- p_values_inhospital_mort
ality.ipynb , Jupyter, 106 lines - p_values_phenotyping.ipy
nb , Jupyter, 104 lines - patient_journey.ipynb, Jupyter, 215 lines, 1 match
- plot_ecgs.ipynb, Jupyter, 74 lines
- plot_labs.ipynb, Jupyter, 109 lines
- plot_meds.ipynb, Jupyter, 312 lines
- utils.py, Python, 200 lines
- README.md, Text, 178 lines
Code availability
Following the FAIR criteria (findability, accessibility, interoperability, and reusability) in scientific research all code used in this study will be made publicly available. The code for cohort creation, data preprocessing, i.e., plot generation, training, and evaluation is open source available at https://
Reproduced under the paper's license (CC BY), from the paper cited above.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 32 scripts, each with its path and the digest of its content;
- 7 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Datasets cited
- doi:10.13026/
1176-6c44 , at the source; found in the references - doi:10.13026/
4jqj-jw95 , at the source; found in the references - doi:10.13026/
4nqg-sb35 , at the source; found in the references - doi:10.13026/
ef48-v217 , at the source; found in the references - physionet.org/
content/ , at PhysioNet; found in “Data availability”covid-data-shared-learni ng - physionet.org/
content/ , at PhysioNet; found in “Data availability”mimic-cxr-jpg - physionet.org/
content/ , at PhysioNet; found in “Data availability”mimic-iv-ecg - physionet.org/
content/ , at PhysioNet; found in “Data availability”mimiciv
Data Availability Statement
All data is publicly available from the MIMIC database22,34 on PhysioNet (MIMICIV: https://
Following the FAIR criteria (findability, accessibility, interoperability, and reusability) in scientific research all code used in this study will be made publicly available. The code for cohort creation, data preprocessing, i.e., plot generation, training, and evaluation is open source available at https://
Reproduced under the paper's license (CC BY), from the paper cited above.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 3, 28 September 2026
- Funding: added Deutsche Forschungsgemeinschaft: INST 35/1314-1 FUGG, INST35/1503‐1FUGG, INST 35/1314‐1, 35/1503-1, 35/1503-1 FUGG, 35/1314-1 FUGG, INST 35, INST 35/1503‐1, 35/1314-1
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 10 authors, 2 keywords, 40 references.
Cite
This paper
Tölle, M., Scharaf, M., Fischer, S., Reich, C., Zeid, S., Dieterich, C., Meder, B., Frey, N., Wild, P., & Engelhardt, S. (2026). Visual prompt engineering for multimodal and irregularly sampled medical data. Communications medicine, 6(1), 474. https://
BibTeX
@article{tolle2026visual
author = {Tölle, Malte and Scharaf, Mohamad and Fischer, Samantha and Reich, Christoph and Zeid, Silav and Dieterich, Christoph and Meder, Benjamin and Frey, Norbert and Wild, Philipp and Engelhardt, Sandy},
title = {{Visual prompt engineering for multimodal and irregularly sampled medical data}},
journal = {Communications medicine},
year = {2026},
month = sep,
volume = {6},
number = {1},
pages = {474},
publisher = {Nature Publishing Group},
issn = {2730-664X},
doi = {10.1038/
url = {https://
pmid = {42680823},
pmcid = {PMC13534428}
}
RIS
TY - JOUR
AU - Tölle, Malte
AU - Scharaf, Mohamad
AU - Fischer, Samantha
AU - Reich, Christoph
AU - Zeid, Silav
AU - Dieterich, Christoph
AU - Meder, Benjamin
AU - Frey, Norbert
AU - Wild, Philipp
AU - Engelhardt, Sandy
TI - Visual prompt engineering for multimodal and irregularly sampled medical data
T2 - Communications medicine
J2 - Commun Med (Lond)
PY - 2026
DA - 2026/
VL - 6
IS - 1
SP - 474
SN - 2730-664X
PB - Nature Publishing Group
DO - 10.1038/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1038/
"type": "article-journal",
"title": "Visual prompt engineering for multimodal and irregularly sampled medical data",
"container-title": "Communications medicine",
"author": [
{
"family": "Tölle",
"given": "Malte"
},
{
"family": "Scharaf",
"given": "Mohamad"
},
{
"family": "Fischer",
"given": "Samantha"
},
{
"family": "Reich",
"given": "Christoph"
},
{
"family": "Zeid",
"given": "Silav"
},
{
"family": "Dieterich",
"given": "Christoph"
},
{
"family": "Meder",
"given": "Benjamin"
},
{
"family": "Frey",
"given": "Norbert"
},
{
"family": "Wild",
"given": "Philipp"
},
{
"family": "Engelhardt",
"given": "Sandy"
}
],
"container-title-short":
"volume": "6",
"issue": "1",
"page": "474",
"DOI": "10.1038/
"PMID": "42680823",
"PMCID": "PMC13534428",
"ISSN": "2730-664X",
"publisher": "Nature Publishing Group",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
9,
1
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1016/j.isci.2026.116135 [code]
- Predicting ICU in-hospital mortality from text-encoded structured EHR data using adaptive transformer layer fusion.Journal: iScienceIn common: Hugging Face Transformers, PyTorch, seaborn, 4 other tools, physionet.org/content/mimiciv, clinical / translational, 3 references
- [2] doi:10.3390/bioengineering12060578 [code]
- PathCare: Integrating Clinical Pathway Information to Enable Healthcare Prediction at the Neuron Level.Journal: Bioengineering (Basel, Switzerland)In common: PyTorch, seaborn, scikit-learn, 3 other tools, physionet.org/content/mimiciv, clinical / translational, 3 references
- [3] doi:10.1038/s41598-026-36296-6
- Integrative multimodal hybrid data fusion for mortality prediction.Journal: Scientific reportsIn common: physionet.org/content/mimic-iv-ecg, physionet.org/content/mimiciv, clinical / translational, 2 references
- [4] doi:10.1038/s41746-025-01773-1 [code]
- Detection of epileptic spasms using foundational AI and smartphone videos.Journal: NPJ digital medicineIn common: Hugging Face Transformers, OpenCV, Pillow, 6 other tools, clinical / translational, 2 references
- [5] doi:10.1016/j.ebiom.2024.105047 [code]
- Drop the shortcuts: image augmentation improves fairness and decreases AI detection of race and other demographics from medical images.Journal: EBioMedicineIn common: OpenCV, Pillow, PyTorch, 5 other tools, 3 references
- [6] doi:10.1016/j.patter.2025.101325 [code]
- Tucano: Advancing neural text generation for Portuguese.Journal: Patterns (New York, N.Y.)In common: Hugging Face Transformers, Pillow, PyTorch, 5 other tools, 2 references
- [7] doi:10.1038/s41746-025-02018-x [code]
- GlioSurv: interpretable transformer for multimodal, individualized survival prediction in diffuse glioma.Journal: NPJ digital medicineIn common: Hugging Face Transformers, PyTorch, pandas, 3 other tools, clinical / translational, 3 references
- [8] doi:10.1097/js9.0000000000003143 [code]
- Multimodal deep learning to predict postoperative major adverse cardiac and cerebrovascular events after noncardiac surgery.Journal: International journal of surgery (London, England)In common: PyTorch, seaborn, scikit-learn, 4 other tools, clinical / translational, 3 references
- [9] doi:10.1371/journal.pone.0344600 [code]
- Robust disease prognosis via diagnostic knowledge preservation: A sequential learning approach.Journal: PloS oneIn common: OpenCV, Pillow, PyTorch, 5 other tools, clinical / translational, 2 references
- [10] doi:10.1093/braincomms/fcag253 [code]
- Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study.Journal: Brain communicationsIn common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools, clinical / translational
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 2 repositories of the authors' code, each at its verified commit and with its license, 32 scripts, and 7 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:064ecc480c9edd79…
Add the badge to the code's README
If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.
Markdown
<!-- oscr-badge:start -->
[](https://openscicode.org/badge/doi/10.1038/s43856-026-01817-x)
<!-- oscr-badge:end -->reStructuredText
.. oscr-badge:start
.. image:: https://openscicode.org/badge/doi/10.1038/s43856-026-01817-x.svg
:target: https://openscicode.org/badge/doi/10.1038/s43856-026-01817-x
:alt: OSCR
.. oscr-badge:endHTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1038/s43856-026-01817-x"><img src="https://openscicode.org/badge/doi/10.1038/s43856-026-01817-x.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".
If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.
Report a mismatch between the code and the paper · Report a code error (signed in)
Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.
Report a reproduction failure (signed in)
Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
