Multi-class classification of brain tumor using a ResNet101 backbone integrated with multi-scale deformable attention module and advanced data augmentations.

Code ↔ Paper

5 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 5 matches

[1] § Results and discussion › Model training, validation and test performance ↔ train-val-test-04-11-2025-16-5.ipynb, lines 472–617 · score 0.73 · cosine annealing learning, rate scheduler, optimization, weights, PyTorch, MixUp
[2] § Results and discussion › Performance evaluation of Precision–Recall, Receiver Operating Characteristic (ROC) curve and AUC & t-SNE feature visualization ↔ train-val-test-04-11-2025-16-5.ipynb, lines 1781–1833 · score 0.56 · PR curves, Precision Recall, ROC, AUC, classifier, model
[3] § Results and discussion › Quantitative evaluation metrics ↔ train-val-test-04-11-2025-16-5.ipynb, lines 1247–1396 · score 0.52 · Cohen Kappa, Macro, Dice, metric, Recall, score
[4] § Methodology › Model architecture › Multi-scale deformable attention module (MS-DAM) ↔ train-val-test-04-11-2025-16-5.ipynb, lines 281–355 · score 0.51 · MS DAM, kernel, bilinear, adaptively, channel, fuse
[5] § Methodology › Dataset preparation › Dataset scanning and deduplication ↔ train-val-test-04-11-2025-16-5.ipynb, lines 103–139 · score 0.50 · brain tumor, download, kagglehub, waseemnagahhenes, MRI, classes

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The paper is loaded by your browser, with JavaScript: read it at doi.org or on Europe PMC.

The authors' code

Jupyter notebook · 2,512 lines · 92 KB · no license · 5 matches

# %%
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory
import os
for dirname, _, filenames in os.walk('/kaggle/input'):
for filename in filenames:
print(os.path.join(dirname, filename))
# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All"
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
# %%
!pip install grad-cam
import pytorch_grad_cam
print("Grad-CAM is installed and working!")
!pip install "numpy<2.0"
# %%
# ======================== SAFE ENVIRONMENT SETUP ========================
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
import os
import math
import random
import copy
import time
import hashlib
import zipfile
from pathlib import Path
# Limit OpenBLAS / MKL / OMP threads to prevent warnings/hangs
os.environ["OPENBLAS_NUM_THREADS"] = "1"
os.environ["OMP_NUM_THREADS"] = "1"
os.environ["MKL_NUM_THREADS"] = "1"
os.environ["NUMEXPR_NUM_THREADS"] = "1"
os.environ["VECLIB_MAXIMUM_THREADS"] = "1"
os.environ["OMP_DYNAMIC"] = "FALSE"
import numpy as np
import pandas as pd
from PIL import Image
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import (
accuracy_score, precision_score, recall_score, f1_score,
cohen_kappa_score, classification_report, roc_auc_score,
confusion_matrix, ConfusionMatrixDisplay, roc_curve, auc,
precision_recall_curve
)
from sklearn.preprocessing import label_binarize
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from scipy.stats import ttest_rel
try:
import shap
except Exception:
shap = None
import torch
import torch.nn as nn
import torch.nn.functional as F # model/ops functional
torch.set_num_threads(1) # Force PyTorch single-threaded
from torch.utils.data import Dataset, DataLoader, TensorDataset
from torchvision import transforms, models
import torchvision.transforms.functional as TF # transform functional
from tqdm import tqdm
# Grad-CAM optional
GRADCAM_AVAILABLE = False
try:
from pytorch_grad_cam import GradCAM
from pytorch_grad_cam.utils.model_targets import ClassifierOutputTarget
from pytorch_grad_cam.utils.image import show_cam_on_image
GRADCAM_AVAILABLE = True
print("pytorch-gradcam library found. Using it for Grad-CAM visualization.")
except Exception:
print("pytorch-gradcam library not found. Using simple Grad-CAM helper.")
# ======================== GLOBALS / HYPERPARAMS ========================
SEED = 42
random.seed(SEED)
np.random.seed(SEED)
torch.manual_seed(SEED)
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Device:", DEVICE)
RESULTS_DIR = "final_results"
os.makedirs(RESULTS_DIR, exist_ok=True)
# Dataset path (Kaggle input / fallback)
DATA_DIR = None
try:
import kagglehub
try:
DATA_DIR = kagglehub.dataset_download("waseemnagahhenes/brain-tumor-for-14-classes")
print("kagglehub returned:", DATA_DIR)
except Exception:
pass
except Exception:
pass
if not DATA_DIR:
candidates = [
"/kaggle/input/brain-tumor-for-14-classes",
"/kaggle/input/brain-tumor-classification-dataset",
"/kaggle/input/brain-tumor-for-14-classes-mri",
"/kaggle/input"
]
for c in candidates:
if os.path.exists(c):
DATA_DIR = c
break
if DATA_DIR is None:
raise RuntimeError("DATA_DIR not set. Please check dataset.")
BATCH_SIZE = 16
IMG_SIZE = 224
NUM_EPOCHS = 400
LR = 1e-4
NUM_WORKERS = 4 if torch.cuda.is_available() else 0
QUICK_DEBUG = False
TSNE_SAMPLES = 2000 if not QUICK_DEBUG else 200
SHAP_MAX_SAMPLES = 500 if not QUICK_DEBUG else 100
DEGRADE_FACTORS = [0.9, 0.7, 0.5]
PLOT_DPI = 300
MIXUP = True
def safe_classification_report(y_true, y_pred):
"""
Wrapper around sklearn's classification_report that avoids
Precision/F-score warnings by using zero_division=0
"""
return classification_report(y_true, y_pred, output_dict=True, zero_division=0)
# ======================== CELL 3: Utilities ========================
def find_image_dirs(root):
image_dirs = []
for dirpath, dirnames, filenames in os.walk(root):
count = sum(1 for f in filenames if f.lower().endswith(('.png','.jpg','.jpeg','.bmp')))
if count > 0:
image_dirs.append(dirpath)
return image_dirs
def get_patient_id(path):
# path can be full path or filename; use file stem
return Path(path).stem.split('_')[0]
def image_hash(path, resize=(64,64)):
try:
with Image.open(path) as im:
im = im.convert("RGB").resize(resize)
return hashlib.md5(np.array(im).tobytes()).hexdigest()
except Exception:
return None
def get_unique_samples(dataset_path):
seen_hashes = set()
unique_samples = []
for root, _, files in os.walk(dataset_path):
for f in files:
if f.lower().endswith(('.png','.jpg','.jpeg')):
path = os.path.join(root, f)
try:
with Image.open(path) as img:
img = img.convert("RGB")
h = hashlib.md5(img.tobytes()).hexdigest()
except Exception:
continue
if h not in seen_hashes:
seen_hashes.add(h)
label = os.path.basename(root)
unique_samples.append((path, label))
return unique_samples
# ======================== CELL 4: Dataset Class ========================
class MRIDataset(Dataset):
def __init__(self, df, transform=None):
self.df = df
self.transform = transform
def __len__(self):
return len(self.df)
def __getitem__(self, idx):
row = self.df.iloc[idx]
img = Image.open(row['path']).convert("RGB")
if self.transform:
img = self.transform(img)
label = int(row['label'])
return img, label
# ===============================
# ✅ Custom Transform Classes
# ===============================
class RandomIntensityScaling(object):
"""Randomly scales image intensity (brightness) by a random factor."""
def __init__(self, scale_range=(0.9, 1.1), p=0.5):
self.scale_range = scale_range
self.p = p
def __call__(self, img):
# Works on PIL Image or Tensor-compatible input that TF.adjust_brightness accepts
if random.random() < self.p:
scale = random.uniform(*self.scale_range)
img = TF.adjust_brightness(img, scale)
return img
class AddGaussianNoise(object):
"""Adds random Gaussian noise to an image tensor (after ToTensor)."""
def __init__(self, mean=0.0, std=0.01, p=0.5):
self.mean = mean
self.std = std
self.p = p
def __call__(self, img):
if random.random() < self.p:
# If input is a PIL Image, convert to tensor
if not isinstance(img, torch.Tensor):
img = TF.to_tensor(img)
noise = torch.randn_like(img) * self.std + self.mean
img = img + noise
img = torch.clamp(img, 0.0, 1.0)
return img
def __repr__(self):
return f"{self.__class__.__name__}(mean={self.mean}, std={self.std}, p={self.p})"
# ===============================
# ✅ Transform Pipelines
# ===============================
train_transform = transforms.Compose([
transforms.RandomResizedCrop((IMG_SIZE, IMG_SIZE), scale=(0.85, 1.0)),
transforms.RandomHorizontalFlip(p=0.5),
transforms.RandomVerticalFlip(p=0.3),
transforms.RandomRotation(degrees=5),
# Light intensity scaling (mimics scanner variability)
RandomIntensityScaling(p=0.5, scale_range=(0.9, 1.1)),
# Convert to Tensor BEFORE adding Gaussian noise or normalization
transforms.ToTensor(),
# Small Gaussian noise (after tensor conversion)
AddGaussianNoise(mean=0., std=0.005, p=0.3),
# Mild contrast and brightness adjustments (operates on PIL or Tensor-compatible inputs)
transforms.ColorJitter(brightness=0.05, contrast=0.05),
# Normalize (ImageNet stats)
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225]),
# Random Erasing (helps robustness)
transforms.RandomErasing(p=0.1, scale=(0.02, 0.08), ratio=(0.3, 3.3))
])
val_transform = transforms.Compose([
transforms.Resize((IMG_SIZE, IMG_SIZE)),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225])
])
# ======================== CELL 6: MS-DAM Module (kept & used by create_backbone) ========================
class MS_DAM(nn.Module):
def __init__(self, in_channels_list, out_channels=512, sampling_kernel=3, offset_scale=0.15, use_se=True):
super().__init__()
self.in_chs = in_channels_list
self.num_scales = len(in_channels_list)
self.out_ch = out_channels
self.offset_scale = offset_scale
self.proj_convs = nn.ModuleList([
nn.Sequential(
nn.Conv2d(c, out_channels, 1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True)
) for c in in_channels_list
])
self.offset_pred = nn.Sequential(
nn.Conv2d(out_channels * self.num_scales, out_channels, 3, padding=1),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels, 2, 1)
)
self.spatial_attn = nn.Sequential(
nn.Conv2d(out_channels, out_channels // 4, 3, padding=1),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels // 4, 1, 1),
nn.Sigmoid()
)
self.use_se = use_se
if use_se:
self.se_fc = nn.Sequential(
nn.AdaptiveAvgPool2d(1),
nn.Conv2d(out_channels, out_channels // 8, 1),
nn.ReLU(inplace=True),
nn.Conv2d(out_channels // 8, out_channels, 1),
nn.Sigmoid()
)
self.fuse = nn.Sequential(
nn.Conv2d(out_channels, out_channels, 3, padding=1, bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True)
)
def forward(self, feats):
projected = []
sizes = [f.shape[-2:] for f in feats]
areas = [s[0] * s[1] for s in sizes]
ref_idx = int(np.argmax(areas))
ref_h, ref_w = sizes[ref_idx]
for i, f in enumerate(feats):
p = self.proj_convs[i](f)
if (p.shape[-2], p.shape[-1]) != (ref_h, ref_w):
p = F.interpolate(p, size=(ref_h, ref_w), mode='bilinear', align_corners=False)
projected.append(p)
concat = torch.cat(projected, dim=1)
offsets = torch.tanh(self.offset_pred(concat)) * self.offset_scale
B, _, H, W = offsets.shape
yy, xx = torch.meshgrid(
torch.linspace(-1, 1, H, device=offsets.device),
torch.linspace(-1, 1, W, device=offsets.device),
indexing='ij'
)
# base_grid shape: (1, H, W, 2)
base_grid = torch.stack((xx, yy), dim=-1).unsqueeze(0).repeat(B, 1, 1, 1)
offsets_grid = offsets.permute(0, 2, 3, 1)
sampling_grid = (base_grid + offsets_grid).clamp(-1, 1)
sampled = F.grid_sample(concat, sampling_grid, mode='bilinear', padding_mode='border', align_corners=True)
sampled = sampled.view(B, self.num_scales, self.out_ch, H, W).mean(dim=1)
spat = self.spatial_attn(sampled)
feat = sampled * spat
if self.use_se:
ch_att = self.se_fc(feat)
feat = feat * ch_att
# in original code there was an extra feat=feat*ch_att; we avoid duplicating multiplication
feat = self.fuse(feat)
return feat
# ======================== CELL 7: Backbone Wrapper & create_backbone ========================
class ResNetWithMSDAM(nn.Module):
def __init__(self, base_resnet, msdam, num_classes):
super().__init__()
self.conv1 = base_resnet.conv1
self.bn1 = base_resnet.bn1
self.relu = base_resnet.relu
self.maxpool = base_resnet.maxpool
self.layer1 = base_resnet.layer1
self.layer2 = base_resnet.layer2
self.layer3 = base_resnet.layer3
self.layer4 = base_resnet.layer4
self.msdam = msdam
self.classifier = nn.Sequential(
nn.AdaptiveAvgPool2d(1),
nn.Flatten(),
nn.Dropout(0.4),
nn.Linear(msdam.out_ch, num_classes)
)
def forward_features(self, x):
x = self.conv1(x); x = self.bn1(x); x = self.relu(x); x = self.maxpool(x)
c2 = self.layer1(x); c3 = self.layer2(c2); c4 = self.layer3(c3); c5 = self.layer4(c4)
return [c2, c3, c4, c5]
def forward(self, x):
feats = self.forward_features(x)
fused = self.msdam(feats)
out = self.classifier(fused)
return out, fused # Return fused features as well for Grad-CAM
def create_backbone(model_type="resnet101", pretrained=True, num_classes=4):
"""
Create model with MS_DAM + ResNet backbone.
model_type currently supports 'resnet101' (can extend if needed).
"""
if model_type == "resnet101":
base = models.resnet101(weights=models.ResNet101_Weights.DEFAULT if pretrained else None)
in_chs = [256, 512, 1024, 2048]
msdam = MS_DAM(in_chs, out_channels=512, offset_scale=0.12, use_se=True)
model = ResNetWithMSDAM(base, msdam, num_classes=num_classes)
else:
raise ValueError(f"Unknown model_type: {model_type}")
model.to(DEVICE)
return model
# ======================== CELL 8: MixUp Helper ========================
def mixup_data(x, y, alpha=0.1):
if alpha <= 0:
return x, y, 1.0, None
lam = np.random.beta(alpha, alpha)
batch_size = x.size()[0]
index = torch.randperm(batch_size).to(x.device)
mixed_x = lam * x + (1 - lam) * x[index, :]
y_a, y_b = y, y[index]
return mixed_x, (y_a, y_b, lam)
# ======================== CELL 9: Logger ========================
class TrainingLogger:
def __init__(self):
self.train_losses = []
self.val_losses = []
self.train_accs = []
self.val_accs = []
def log_epoch(self, train_loss, val_loss, train_acc, val_acc):
self.train_losses.append(train_loss)
self.val_losses.append(val_loss)
self.train_accs.append(train_acc)
self.val_accs.append(val_acc)
def plot(self):
epochs = range(1, len(self.train_losses) + 1)
plt.figure(figsize=(10, 5))
plt.plot(epochs, self.train_losses, 'b-', label="Train Loss")
plt.plot(epochs, self.val_losses, 'r-', label="Val Loss")
plt.xlabel("Epoch"); plt.ylabel("Loss"); plt.title("Training & Validation Loss"); plt.legend(); plt.show()
plt.figure(figsize=(10, 5))
plt.plot(epochs, self.train_accs, 'b-', label="Train Acc")
plt.plot(epochs, self.val_accs, 'r-', label="Val Acc")
plt.xlabel("Epoch"); plt.ylabel("Accuracy"); plt.title("Training & Validation Accuracy"); plt.legend(); plt.show()
# ======================== CELL 10: Feature Extraction + SVM ========================
def extract_penultimate_features(model, loader):
model.eval()
feats = []
labels = []
with torch.no_grad():
for imgs, lbls in loader:
imgs = imgs.to(DEVICE)
out, fused = model(imgs) # model returns (logits, features)
# pooled fused features
fused_pooled = fused.mean(dim=[2, 3])
feats.append(fused_pooled.cpu().numpy())
labels.append(lbls.numpy())
return np.vstack(feats), np.hstack(labels)
def train_svm_on_features(model, train_loader, val_loader):
X_train, y_train = extract_penultimate_features(model, train_loader)
X_val, y_val = extract_penultimate_features(model, val_loader)
scaler = StandardScaler().fit(X_train)
clf = SVC(kernel='rbf', C=10, gamma='scale')
clf.fit(scaler.transform(X_train), y_train)
acc = clf.score(scaler.transform(X_val), y_val)
print(f"SVM on penultimate features Val Acc: {acc*100:.2f}%")
return clf, scaler
# ======================== CELL 11: Train + Save + Evaluate Best Accuracy & Loss Models ========================
import pandas as _pd # local alias to avoid clobbering pd in outer scope (kept but not necessary)
def train_one_model(model, train_loader, val_loader, test_loader,
epochs=20, lr=1e-4, mixup=True, out_dir="./results"):
"""
Trains a model, saves best models (by val accuracy and val loss),
uses tie-breaking on train metrics, and evaluates both models.
(Early stopping removed)
"""
os.makedirs(out_dir, exist_ok=True)
optimizer = torch.optim.AdamW(model.parameters(), lr=lr, weight_decay=1e-4)
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=epochs, eta_min=1e-6)
criterion = nn.CrossEntropyLoss(label_smoothing=0.1)
logger = TrainingLogger()
best_val_acc = -1.0
best_val_loss = float('inf')
best_train_acc_for_best_val = -1.0
best_train_loss_for_best_val = float('inf')
best_acc_epoch = -1
best_loss_epoch = -1
best_acc_path = os.path.join(out_dir, "best_acc_model.pth")
best_loss_path = os.path.join(out_dir, "best_loss_model.pth")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
print("🚀 Starting Training with Dual Criteria (No Early Stopping)...")
for epoch in range(epochs):
model.train()
running_loss, correct, total = 0.0, 0, 0
# ---------------- Training ----------------
for imgs, labels in train_loader:
imgs, labels = imgs.to(device), labels.to(device)
if mixup:
imgs, (y_a, y_b, lam) = mixup_data(imgs, labels)
out, _ = model(imgs)
loss = lam * criterion(out, y_a) + (1 - lam) * criterion(out, y_b)
else:
out, _ = model(imgs)
loss = criterion(out, labels)
optimizer.zero_grad()
loss.backward()
optimizer.step()
running_loss += loss.item() * imgs.size(0)
_, preds = out.max(1)
if mixup:
correct += lam * (preds == y_a).sum().item() + (1 - lam) * (preds == y_b).sum().item()
else:
correct += (preds == labels).sum().item()
total += imgs.size(0)
train_loss = running_loss / total
train_acc = correct / total
# step scheduler (end of epoch)
scheduler.step()
# ---------------- Validation ----------------
model.eval()
val_loss, correct_val, total_val = 0.0, 0, 0
with torch.no_grad():
for imgs, labels in val_loader:
imgs, labels = imgs.to(device), labels.to(device)
out, _ = model(imgs)
loss = criterion(out, labels)
val_loss += loss.item() * imgs.size(0)
_, preds = out.max(1)
correct_val += (preds == labels).sum().item()
total_val += imgs.size(0)
val_loss /= total_val
val_acc = correct_val / total_val
logger.log_epoch(train_loss, val_loss, train_acc, val_acc)
print(f"Epoch {epoch+1}/{epochs} | "
f"Train Loss {train_loss:.4f} | Val Loss {val_loss:.4f} | "
f"Train Acc {train_acc:.4f} | Val Acc {val_acc:.4f}")
# ---------------- Save Best Models ----------------
# Best by val accuracy (tie-breaker: higher train acc)
if (val_acc > best_val_acc) or (abs(val_acc - best_val_acc) < 1e-6 and train_acc > best_train_acc_for_best_val):
best_val_acc = val_acc
best_train_acc_for_best_val = train_acc
best_acc_epoch = epoch + 1
torch.save(model.state_dict(), best_acc_path)
print(f"✅ Saved Best Model (Val Acc): Epoch {best_acc_epoch} | Val Acc {best_val_acc:.4f}")
# Best by val loss (tie-breaker: lower train loss)
if (val_loss < best_val_loss) or (abs(val_loss - best_val_loss) < 1e-6 and train_loss < best_train_loss_for_best_val):
best_val_loss = val_loss
best_train_loss_for_best_val = train_loss
best_loss_epoch = epoch + 1
torch.save(model.state_dict(), best_loss_path)
print(f"✅ Saved Best Model (Val Loss): Epoch {best_loss_epoch} | Val Loss {best_val_loss:.4f}")
# ---------------- After Training ----------------
print("\n🏁 Training Complete!")
print(f"✅ Best Model (Val Acc): Epoch {best_acc_epoch} | Best Val Acc: {best_val_acc:.4f}")
print(f"✅ Best Model (Val Loss): Epoch {best_loss_epoch} | Best Val Loss: {best_val_loss:.4f}")
# ---------------- Save Training History ----------------
history = pd.DataFrame({
"epoch": range(1, len(logger.train_losses) + 1),
"train_loss": logger.train_losses,
"val_loss": logger.val_losses,
"train_acc": logger.train_accs,
"val_acc": logger.val_accs
})
history.to_csv(os.path.join(out_dir, "history.csv"), index=False)
# ---------------- Plot Loss & Accuracy ----------------
def save_plot(metric_name, train_values, val_values):
plt.figure(figsize=(10, 4))
plt.plot(train_values, "b-", label=f"Train {metric_name}", linewidth=2)
plt.plot(val_values, "r-", label=f"Val {metric_name}", linewidth=2)
plt.title(f"{metric_name} vs Epoch")
plt.xlabel("Epoch"); plt.ylabel(metric_name)
plt.legend(); plt.grid(True)
plt.tight_layout()
plt.savefig(os.path.join(out_dir, f"{metric_name.lower()}_plot.png"), dpi=200)
plt.close()
save_plot("Accuracy", logger.train_accs, logger.val_accs)
save_plot("Loss", logger.train_losses, logger.val_losses)
# ---------------- Load Best Models ----------------
model_acc = copy.deepcopy(model)
model_acc.load_state_dict(torch.load(best_acc_path, map_location=device))
model_loss = copy.deepcopy(model)
model_loss.load_state_dict(torch.load(best_loss_path, map_location=device))
# ---------------- Evaluate Both Models ----------------
for name, model_sel in zip(["accuracy_model", "loss_model"], [model_acc, model_loss]):
sub_out_dir = os.path.join(out_dir, name)
os.makedirs(sub_out_dir, exist_ok=True)
evaluate_on_loader(model_sel, test_loader, criterion, out_dir=sub_out_dir)
return model_acc, model_loss, logger
def evaluate_on_loader(model, loader, criterion, out_dir=None):
import os
import pandas as pd
import torch.nn.functional as F_local
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
model.eval()
y_true, y_pred, y_prob = [], [], []
total_loss = 0.0
with torch.no_grad():
for imgs, labels in loader:
imgs, labels = imgs.to(DEVICE), labels.to(DEVICE)
out, _ = model(imgs) # model returns (logits, features)
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
probs = F_local.softmax(out, dim=1).cpu().detach().numpy()
preds = probs.argmax(axis=1)
y_true.extend(labels.cpu().numpy())
y_pred.extend(preds)
y_prob.extend(probs)
# Convert to arrays
y_true, y_pred, y_prob = np.array(y_true), np.array(y_pred), np.array(y_prob)
# Metrics
avg_loss = total_loss / len(loader.dataset)
acc = accuracy_score(y_true, y_pred)
if out_dir:
os.makedirs(out_dir, exist_ok=True)
# 1️⃣ Save basic metrics to CSV
metrics_df = pd.DataFrame([{
"loss": avg_loss,
"accuracy": acc,
"num_samples": len(y_true)
}])
metrics_df.to_csv(os.path.join(out_dir, "metrics.csv"), index=False)
# 2️⃣ Save classification report to CSV
cls_report_dict = classification_report(y_true, y_pred, output_dict=True, zero_division=0)
cls_report_df = pd.DataFrame(cls_report_dict).transpose()
cls_report_df.to_csv(os.path.join(out_dir, "classification_report.csv"))
# 3️⃣ Save confusion matrix to CSV
cm = confusion_matrix(y_true, y_pred)
cm_df = pd.DataFrame(cm, index=[str(i) for i in range(len(cm))],
columns=[str(i) for i in range(len(cm))])
cm_df.to_csv(os.path.join(out_dir, "confusion_matrix.csv"))
print(f"✅ Evaluation metrics saved to {out_dir} (metrics.csv, classification_report.csv, confusion_matrix.csv)")
return avg_loss, acc, y_true, y_pred, y_prob
# ======================== Additional reporting helpers ========================
def save_reports(y_true, y_pred, class_names, out_dir):
cm = confusion_matrix(y_true, y_pred)
df_cm = pd.DataFrame(cm, index=class_names, columns=class_names)
df_cm.to_csv(os.path.join(out_dir, "confusion_matrix.csv"))
plt.figure(figsize=(10, 8))
sns.heatmap(df_cm, annot=True, fmt="d", cmap="Blues")
plt.title("Confusion Matrix"); plt.ylabel("True"); plt.xlabel("Predicted")
plt.tight_layout(); plt.savefig(os.path.join(out_dir, "confusion_matrix.png"), dpi=PLOT_DPI)
plt.close()
report = classification_report(y_true, y_pred, target_names=class_names, output_dict=True, zero_division=0)
pd.DataFrame(report).to_csv(os.path.join(out_dir, "classification_report.csv"))
return df_cm, report
def save_misclassified(df_test, y_true, y_pred, class_names, out_dir):
mis_idx = np.where(y_true != y_pred)[0]
mis_list = [(df_test.iloc[i]['path'], class_names[y_true[i]], class_names[y_pred[i]]) for i in mis_idx]
pd.DataFrame(mis_list, columns=["image_path", "true_label", "pred_label"]).to_csv(
os.path.join(out_dir, "misclassified.csv"), index=False)
# ======================== DATA SCAN, SPLIT, DATALOADERS ========================
print("🔍 Scanning dataset...")
unique_samples = get_unique_samples(DATA_DIR)
class_names = sorted({label for _, label in unique_samples})
class_to_idx = {c: i for i, c in enumerate(class_names)}
labels_map_inv = {i: c for c, i in class_to_idx.items()} # for Grad-CAM
numeric_samples = [(p, class_to_idx[lab], get_patient_id(p)) for p, lab in unique_samples]
df = pd.DataFrame(numeric_samples, columns=['path', 'label', 'pid'])
# Split by patient
patient_ids = df['pid'].unique()
random.shuffle(patient_ids)
n_train, n_val = int(0.7 * len(patient_ids)), int(0.15 * len(patient_ids))
train_pids = patient_ids[:n_train]
val_pids = patient_ids[n_train:n_train + n_val]
test_pids = patient_ids[n_train + n_val:]
df_train = df[df['pid'].isin(train_pids)].reset_index(drop=True)
df_val = df[df['pid'].isin(val_pids)].reset_index(drop=True)
df_test = df[df['pid'].isin(test_pids)].reset_index(drop=True)
print(f"✅ Samples — Train: {len(df_train)}, Val: {len(df_val)}, Test: {len(df_test)}")
# Count samples and unique patients per split
splits = [("Train", df_train), ("Val", df_val), ("Test", df_test)]
print("📊 Dataset summary:")
print("{:<8} {:>10} {:>15}".format("Split", "Samples", "Unique Patients"))
print("-" * 35)
for name, df_split in splits:
n_samples = len(df_split)
n_patients = df_split['pid'].nunique()
print(f"{name:<8} {n_samples:>10} {n_patients:>15}")
# Optional: check patient overlap between splits
train_pids_set = set(df_train['pid'])
val_pids_set = set(df_val['pid'])
test_pids_set = set(df_test['pid'])
print("\n🧪 Patient overlaps (should be 0):")
print("Train ∩ Val:", len(train_pids_set & val_pids_set))
print("Train ∩ Test:", len(train_pids_set & test_pids_set))
print("Val ∩ Test:", len(val_pids_set & test_pids_set))
print("🧠 Creating DataLoaders...")
train_loader = DataLoader(MRIDataset(df_train, train_transform),
batch_size=BATCH_SIZE, shuffle=True, num_workers=NUM_WORKERS)
val_loader = DataLoader(MRIDataset(df_val, val_transform),
batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS)
test_loader = DataLoader(MRIDataset(df_test, val_transform),
batch_size=BATCH_SIZE, shuffle=False, num_workers=NUM_WORKERS)
print("✅ DataLoaders ready.")
# ======================== CREATE MODEL, TRAIN ========================
print("⚙️ Creating model...")
model = create_backbone("resnet101", num_classes=len(class_names))
print("✅ Model created.")
print("🚀 Starting training...")
model_acc, model_loss, logger = train_one_model(
model=model,
train_loader=train_loader,
val_loader=val_loader,
test_loader=test_loader,
epochs=NUM_EPOCHS,
lr=LR,
mixup=MIXUP,
out_dir=RESULTS_DIR,
)
print("✅ Training completed.")
# %%
# Paths for both best models
best_acc_path = os.path.join(RESULTS_DIR, "best_acc_model.pth")
best_loss_path = os.path.join(RESULTS_DIR, "best_loss_model.pth")
# Dictionary to store the loaded models
best_models = {}
# Load best accuracy model
if os.path.exists(best_acc_path):
model_acc = create_backbone("resnet101", num_classes=len(class_names)) # new model instance
model_acc.load_state_dict(torch.load(best_acc_path, map_location=DEVICE))
model_acc.to(DEVICE)
model_acc.eval()
best_models['best_acc'] = model_acc
print("✅ Loaded Best Accuracy Model:", best_acc_path)
else:
print("⚠️ Best accuracy model not found:", best_acc_path)
# Load best loss model
if os.path.exists(best_loss_path):
model_loss = create_backbone("resnet101", num_classes=len(class_names)) # new model instance
model_loss.load_state_dict(torch.load(best_loss_path, map_location=DEVICE))
model_loss.to(DEVICE)
model_loss.eval()
best_models['best_loss'] = model_loss
print("✅ Loaded Best Loss Model:", best_loss_path)
else:
print("⚠️ Best loss model not found:", best_loss_path)
import os
import matplotlib.pyplot as plt
from sklearn.metrics import roc_curve, auc, precision_recall_curve
from sklearn.preprocessing import label_binarize
def plot_roc_pr(y_true, y_prob, class_names, out_dir, plot_dpi=300):
os.makedirs(out_dir, exist_ok=True)
y_bin = label_binarize(y_true, classes=list(range(len(class_names))))
# ---- ROC Curves ----
plt.figure(figsize=(10,8))
for i, c in enumerate(class_names):
fpr, tpr, _ = roc_curve(y_bin[:, i], y_prob[:, i])
roc_auc = auc(fpr, tpr)
plt.plot(fpr, tpr, label=f"{c} (AUC={roc_auc:.2f})")
plt.plot([0, 1], [0, 1], 'k--')
plt.xlabel("False Positive Rate (FPR)")
plt.ylabel("True Positive Rate (TPR)")
plt.title("ROC Curves (All Classes)")
plt.legend()
plt.tight_layout()
roc_path = os.path.join(out_dir, "roc_curves.png")
plt.savefig(roc_path, dpi=plot_dpi)
plt.close()
# ---- Precision–Recall Curves ----
plt.figure(figsize=(10,8))
for i, c in enumerate(class_names):
prec, rec, _ = precision_recall_curve(y_bin[:, i], y_prob[:, i])
plt.plot(rec, prec, label=c)
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.title("Precision–Recall Curves (All Classes)")
plt.legend()
plt.tight_layout()
pr_path = os.path.join(out_dir, "pr_curves.png")
plt.savefig(pr_path, dpi=plot_dpi)
plt.close()
print(f"✅ ROC and PR plots saved successfully:\n"
f" • {roc_path}\n"
f" • {pr_path}")
eval_out_dir = os.path.join(RESULTS_DIR, "evaluation")
os.makedirs(eval_out_dir, exist_ok=True)
for model_key, model_instance in best_models.items():
# 1️⃣ Create model-specific output folder
model_out_dir = os.path.join(eval_out_dir, model_key)
os.makedirs(model_out_dir, exist_ok=True)
# 2️⃣ Get predictions and probabilities for this model
all_labels, all_probs = [], []
model_instance.eval()
with torch.no_grad():
for imgs, labels in test_loader:
imgs = imgs.to(DEVICE)
labels = labels.to(DEVICE)
outputs, _ = model_instance(imgs) # unpack tuple if model returns (logits, features)
probs = torch.softmax(outputs, dim=1)
all_labels.extend(labels.cpu().numpy())
all_probs.extend(probs.cpu().numpy())
y_true_model = np.array(all_labels)
y_prob_model = np.array(all_probs)
# 3️⃣ Call your plotting function
plot_roc_pr(y_true_model, y_prob_model, class_names, model_out_dir)
from sklearn.manifold import TSNE
def save_evaluation_reports(y_true, y_pred, y_prob, class_names, out_dir):
# Confusion matrix
cm = confusion_matrix(y_true, y_pred)
df_cm = pd.DataFrame(cm, index=class_names, columns=class_names)
df_cm.to_csv(os.path.join(out_dir, "confusion_matrix.csv"))
# Classification report
report = classification_report(y_true, y_pred, target_names=class_names, output_dict=True)
report_df = pd.DataFrame(report).transpose()
report_df.to_csv(os.path.join(out_dir, "classification_report.csv"))
return df_cm, report_df
def evaluate_on_loader(model, loader, criterion, device=DEVICE):
model.to(device)
model.eval()
y_true, y_pred, y_prob = [], [], []
total_loss = 0.0
with torch.no_grad():
for imgs, labels in loader:
imgs, labels = imgs.to(device), labels.to(device)
out = model(imgs)
if isinstance(out, (tuple, list)):
out = out[0] # logits
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
probs = torch.softmax(out, dim=1).cpu().numpy()
preds = np.argmax(probs, axis=1)
y_true.extend(labels.cpu().numpy())
y_pred.extend(preds)
y_prob.extend(probs)
avg_loss = total_loss / len(loader.dataset)
acc = accuracy_score(y_true, y_pred)
return avg_loss, acc, np.array(y_true), np.array(y_pred), np.array(y_prob)
def save_misclassified(df_test, y_true, y_pred, class_names, out_dir):
mis_idx = np.where(y_true != y_pred)[0]
mis_list = [(df_test.iloc[i]['path'], class_names[y_true[i]], class_names[y_pred[i]]) for i in mis_idx]
df_mis = pd.DataFrame(mis_list, columns=["image_path", "true_label", "pred_label"])
df_mis.to_csv(os.path.join(out_dir, "misclassified.csv"), index=False)
return df_mis
def plot_tsne_features(model, loader, class_names, out_dir, n_samples=100):
X, y = extract_penultimate_features(model, loader)
if len(X) > n_samples:
idx = np.random.choice(len(X), n_samples, replace=False)
X, y = X[idx], y[idx]
tsne = TSNE(n_components=2, random_state=42)
X_tsne = tsne.fit_transform(X)
plt.figure(figsize=(8,6))
for cls in np.unique(y):
plt.scatter(X_tsne[y==cls,0], X_tsne[y==cls,1], label=class_names[cls], alpha=0.7)
plt.legend(); plt.title("t-SNE of Penultimate Features")
plt.savefig(os.path.join(out_dir, "tsne_features_1.png"), dpi=300)
plt.close()
# -----------------------------
# Multi-model evaluation loop
# -----------------------------
for name, model in best_models.items():
print(f"\n🧪 Evaluating {name} model...")
eval_out_dir = os.path.join(RESULTS_DIR, f"evaluation_{name}")
Path(eval_out_dir).mkdir(parents=True, exist_ok=True)
# 1️⃣ SVM on penultimate features
print("🧩 Training SVM on penultimate features...")
model.to(DEVICE)
clf, scaler = train_svm_on_features(model, train_loader, val_loader)
# 2️⃣ Evaluate on test set
print("📊 Generating evaluation reports...")
test_loss, test_acc, y_true, y_pred, y_prob = evaluate_on_loader(model, test_loader, nn.CrossEntropyLoss(), device=DEVICE)
print(f"✅ {name} Test Loss: {test_loss:.4f} | Accuracy: {test_acc:.4f}")
# 3️⃣ Save reports
df_cm, report_df = save_evaluation_reports(y_true, y_pred, y_prob, class_names, eval_out_dir)
# 5️⃣ Misclassified samples
df_mis = save_misclassified(df_test, y_true, y_pred, class_names, eval_out_dir)
# 6️⃣ t-SNE visualization
plot_tsne_features(model, test_loader, class_names, eval_out_dir, n_samples=TSNE_SAMPLES)
print(f"\n✅ Evaluation completed for {name} model. Reports saved to {eval_out_dir}")
import os
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt
from pathlib import Path
from sklearn.metrics import confusion_matrix, classification_report, accuracy_score
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA
# -----------------------------------------------------------
# ✅ Helper 1: Extract penultimate features
# -----------------------------------------------------------
def extract_penultimate_features(model, loader, device="cuda"):
"""
Extracts penultimate layer features for visualization.
Automatically flattens spatial dimensions if needed.
Assumes model returns (logits, features) in its forward pass.
"""
model.eval()
features, labels = [], []
with torch.no_grad():
for imgs, lbls in loader:
imgs = imgs.to(device)
lbls = lbls.to(device)
out = model(imgs)
if isinstance(out, (tuple, list)):
logits, feats = out # (logits, features)
else:
feats = out # in case model outputs only features
feats = feats.cpu().numpy()
# 🔧 Flatten 4D tensors [B, C, H, W] → [B, C*H*W]
if feats.ndim > 2:
feats = feats.reshape(feats.shape[0], -1)
features.append(feats)
labels.extend(lbls.cpu().numpy())
return np.concatenate(features), np.array(labels)
# -----------------------------------------------------------
# ✅ Helper 2: Evaluation on test/val loader
# -----------------------------------------------------------
def evaluate_on_loader(model, loader, criterion, device="cuda"):
model.to(device)
model.eval()
y_true, y_pred, y_prob = [], [], []
total_loss = 0.0
with torch.no_grad():
for imgs, labels in loader:
imgs, labels = imgs.to(device), labels.to(device)
out = model(imgs)
if isinstance(out, (tuple, list)):
out = out[0] # logits
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
probs = F.softmax(out, dim=1).cpu().numpy()
preds = np.argmax(probs, axis=1)
y_true.extend(labels.cpu().numpy())
y_pred.extend(preds)
y_prob.extend(probs)
avg_loss = total_loss / len(loader.dataset)
acc = accuracy_score(y_true, y_pred)
return avg_loss, acc, np.array(y_true), np.array(y_pred), np.array(y_prob)
# -----------------------------------------------------------
# ✅ Helper 3: Save evaluation reports
# -----------------------------------------------------------
def save_evaluation_reports(y_true, y_pred, y_prob, class_names, out_dir):
os.makedirs(out_dir, exist_ok=True)
# Confusion matrix
cm = confusion_matrix(y_true, y_pred)
df_cm = pd.DataFrame(cm, index=class_names, columns=class_names)
df_cm.to_csv(os.path.join(out_dir, "confusion_matrix.csv"))
# Classification report
report = classification_report(y_true, y_pred, target_names=class_names,
output_dict=True, zero_division=0)
report_df = pd.DataFrame(report).transpose()
report_df.to_csv(os.path.join(out_dir, "classification_report.csv"))
print("✅ Saved confusion matrix & classification report")
return df_cm, report_df
# -----------------------------------------------------------
# ✅ Helper 4: Save misclassified samples
# -----------------------------------------------------------
def save_misclassified(y_true, y_pred, class_names, out_dir, df_test=None):
mis_idx = np.where(y_true != y_pred)[0]
if df_test is not None and "path" in df_test.columns:
mis_list = [(df_test.iloc[i]['path'], class_names[y_true[i]], class_names[y_pred[i]]) for i in mis_idx]
df_mis = pd.DataFrame(mis_list, columns=["image_path", "true_label", "pred_label"])
else:
df_mis = pd.DataFrame({
"index": mis_idx,
"true_label": [class_names[y_true[i]] for i in mis_idx],
"pred_label": [class_names[y_pred[i]] for i in mis_idx]
})
os.makedirs(out_dir, exist_ok=True)
df_mis.to_csv(os.path.join(out_dir, "misclassified.csv"), index=False)
print(f"✅ Saved misclassified samples CSV ({len(df_mis)} samples)")
return df_mis
# -----------------------------------------------------------
# ✅ Helper 5: Plot t-SNE with PCA preprocessing
# -----------------------------------------------------------
def plot_tsne_features(model, loader, class_names, out_dir, n_samples=300, device="cuda"):
X, y = extract_penultimate_features(model, loader, device=device)
if len(X) > n_samples:
idx = np.random.choice(len(X), n_samples, replace=False)
X, y = X[idx], y[idx]
# PCA to 50D before t-SNE (speeds up & denoises)
pca = PCA(n_components=min(50, X.shape[1]))
X_pca = pca.fit_transform(X)
tsne = TSNE(n_components=2, random_state=42, perplexity=30)
X_tsne = tsne.fit_transform(X_pca)
plt.figure(figsize=(8, 6))
for cls in np.unique(y):
plt.scatter(X_tsne[y == cls, 0], X_tsne[y == cls, 1],
label=class_names[cls], alpha=0.7, s=40)
plt.legend()
plt.title("t-SNE of Penultimate Features (PCA Preprocessed)")
plt.tight_layout()
plt.savefig(os.path.join(out_dir, "tsne_features.png"), dpi=300)
plt.close()
print("✅ Saved t-SNE visualization")
# -----------------------------------------------------------
# ✅ Main Evaluation Loop for Multiple Models
# -----------------------------------------------------------
def evaluate_all_models(best_models, train_loader, val_loader, test_loader,
RESULTS_DIR, class_names, TSNE_SAMPLES=300, DEVICE="cuda",
df_test=None):
criterion = nn.CrossEntropyLoss()
for name, model in best_models.items():
print(f"\n🧪 Evaluating {name} model...")
eval_out_dir = os.path.join(RESULTS_DIR, f"evaluation_{name}")
Path(eval_out_dir).mkdir(parents=True, exist_ok=True)
# 1️⃣ Evaluate performance
print("📊 Evaluating model on test set...")
test_loss, test_acc, y_true, y_pred, y_prob = evaluate_on_loader(
model, test_loader, criterion, device=DEVICE)
print(f"✅ {name} Test Loss: {test_loss:.4f} | Accuracy: {test_acc:.4f}")
# 2️⃣ Save evaluation reports
df_cm, report_df = save_evaluation_reports(y_true, y_pred, y_prob, class_names, eval_out_dir)
# 3️⃣ Save misclassified samples
df_mis = save_misclassified(y_true, y_pred, class_names, eval_out_dir, df_test=df_test)
# 4️⃣ Plot t-SNE of learned features
plot_tsne_features(model, test_loader, class_names, eval_out_dir,
n_samples=TSNE_SAMPLES, device=DEVICE)
print(f"✅ Evaluation completed for {name} model. Reports saved to {eval_out_dir}\n")
evaluate_all_models(
best_models=best_models,
train_loader=train_loader,
val_loader=val_loader,
test_loader=test_loader,
RESULTS_DIR=RESULTS_DIR,
class_names=class_names,
TSNE_SAMPLES=2000,
DEVICE=DEVICE,
df_test=df_test # optional, if you have test dataframe with image paths
)
def plot_samples_per_class_fixed_cols(model, df_test, class_names, out_dir,
samples_per_class=2, device='cuda',
random_state=42, img_size=224,
images_per_row=5):
"""
Shows test images with true/pred labels, arranged with a fixed number of images per row.
Handles models that return a tuple (logits, features) or just logits.
"""
os.makedirs(out_dir, exist_ok=True)
model.eval()
model.to(device)
preprocess = transforms.Compose([
transforms.Resize((img_size, img_size)),
transforms.ToTensor(),
transforms.Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])
])
# Select samples
np.random.seed(random_state)
selected_dfs = []
for cls_idx in range(len(class_names)):
class_subset = df_test[df_test['label'] == cls_idx]
selected = class_subset.sample(
n=min(samples_per_class, len(class_subset)), random_state=random_state)
selected_dfs.append(selected)
sample_df = pd.concat(selected_dfs, ignore_index=True)
total_samples = len(sample_df)
cols = images_per_row
rows = int(np.ceil(total_samples / cols))
fig, axes = plt.subplots(rows, cols, figsize=(cols * 3.5, rows * 3.5))
axes = np.array(axes).reshape(rows, cols)
for idx, (_, row) in enumerate(sample_df.iterrows()):
r = idx // cols
c = idx % cols
img_path = row['path']
true_label = class_names[int(row['label'])]
try:
img = Image.open(img_path).convert("RGB")
img_tensor = preprocess(img).unsqueeze(0).to(device)
with torch.no_grad():
output = model(img_tensor)
if isinstance(output, tuple):
output = output[0]
prob = torch.softmax(output, dim=1)
pred = torch.argmax(prob, dim=1).item()
pred_label = class_names[pred]
conf = prob[0, pred].item() * 100
axes[r, c].imshow(img)
color = 'green' if pred_label == true_label else 'red'
axes[r, c].set_title(f"{true_label}\n→ {pred_label} ({conf:.1f}%)",
fontsize=14, color=color)
axes[r, c].axis('off')
except Exception as e:
axes[r, c].set_title("Error", fontsize=8)
axes[r, c].axis('off')
print(f"⚠️ Could not load image {img_path}: {e}")
# Turn off remaining empty axes
for idx in range(total_samples, rows * cols):
r = idx // cols
c = idx % cols
axes[r, c].axis('off')
plt.tight_layout()
save_path = os.path.join(out_dir, f"samples_per_class_{samples_per_class}_4perrow.png")
plt.savefig(save_path, dpi=300)
plt.close()
print(f"✅ Saved plot → {save_path}")
for model_key, model_instance in best_models.items():
model_out_dir = os.path.join(RESULTS_DIR, "evaluation", model_key)
os.makedirs(model_out_dir, exist_ok=True)
print(f"\n📊 Plotting 6x5 sample images for model: {model_key}")
plot_samples_per_class_fixed_cols(
model=model_instance,
df_test=df_test,
class_names=class_names,
out_dir=model_out_dir,
samples_per_class=2,
device='cuda',
images_per_row=5
)
import os
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
from sklearn.metrics import confusion_matrix, cohen_kappa_score
import matplotlib.pyplot as plt
from PIL import Image
from torchvision import transforms
def evaluate_full_pipeline(model, model_name, loader, df_test, class_names, out_dir,
device='cuda', criterion=None, samples_per_class=2, IMG_SIZE=224):
"""
Full evaluation pipeline:
✅ Accuracy, Loss, Per-class metrics, Macro/Micro averages
✅ Confusion matrix & classification report CSV
✅ Misclassified samples & bar plot
✅ Visualization: samples_per_class per class
✅ Robust handling of models returning (logits, features)
"""
os.makedirs(out_dir, exist_ok=True)
model.eval()
model.to(device)
if criterion is None:
criterion = nn.CrossEntropyLoss()
y_true_list, y_pred_list, y_prob_list = [], [], []
total_loss, total_samples = 0.0, 0
# -----------------------------
# 1️⃣ Inference over test set
# -----------------------------
with torch.no_grad():
for imgs, labels in loader:
imgs, labels = imgs.to(device), labels.to(device)
out = model(imgs)
if isinstance(out, (tuple, list)):
out = out[0] # take logits only
probs = torch.softmax(out, dim=1)
preds = torch.argmax(probs, dim=1)
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
total_samples += imgs.size(0)
y_true_list.extend(labels.cpu().numpy())
y_pred_list.extend(preds.cpu().numpy())
y_prob_list.extend(probs.cpu().numpy())
y_true = np.array(y_true_list)
y_pred = np.array(y_pred_list)
y_prob = np.array(y_prob_list)
avg_loss = total_loss / total_samples
acc = (y_true == y_pred).mean()
# -----------------------------
# 2️⃣ Confusion Matrix
# -----------------------------
cm = confusion_matrix(y_true, y_pred)
cm_df = pd.DataFrame(cm, index=class_names, columns=class_names)
cm_df.to_csv(os.path.join(out_dir, "confusion_matrix.csv"))
# -----------------------------
# 3️⃣ Per-Class Metrics
# -----------------------------
per_class_metrics = []
for i, cname in enumerate(class_names):
TP = cm[i, i]
FP = cm[:, i].sum() - TP
FN = cm[i, :].sum() - TP
TN = cm.sum() - (TP + FP + FN)
acc_cls = (TP + TN) / cm.sum() if cm.sum() > 0 else 0
prec = TP / (TP + FP) if (TP + FP) > 0 else 0
rec = TP / (TP + FN) if (TP + FN) > 0 else 0
f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
dice = 2 * TP / (2 * TP + FP + FN) if (2 * TP + FP + FN) > 0 else 0
spec = TN / (TN + FP) if (TN + FP) > 0 else 0
fdr = FP / (TP + FP) if (TP + FP) > 0 else 0
forr = FN / (FN + TN) if (FN + TN) > 0 else 0
y_true_bin = (y_true == i).astype(int)
y_pred_bin = (y_pred == i).astype(int)
kappa = cohen_kappa_score(y_true_bin, y_pred_bin)
per_class_metrics.append([cname, acc_cls, prec, rec, f1, dice, spec, fdr, forr, kappa])
metrics_df = pd.DataFrame(
per_class_metrics,
columns=["Class", "Accuracy", "Precision", "Recall", "F1", "Dice",
"Specificity", "FDR", "FOR", "Kappa"]
)
# Macro & Micro averages
macro_avg = metrics_df.iloc[:, 1:].mean().to_dict()
micro_avg = {
"Accuracy": acc,
"Precision": np.mean(metrics_df["Precision"]),
"Recall": np.mean(metrics_df["Recall"]),
"F1": np.mean(metrics_df["F1"]),
"Dice": np.mean(metrics_df["Dice"]),
"Specificity": np.mean(metrics_df["Specificity"]),
"FDR": np.mean(metrics_df["FDR"]),
"FOR": np.mean(metrics_df["FOR"]),
"Kappa": np.mean(metrics_df["Kappa"]),
}
metrics_df = pd.concat([
metrics_df,
pd.DataFrame([["Macro Avg"] + list(macro_avg.values())], columns=metrics_df.columns),
pd.DataFrame([["Micro Avg"] + list(micro_avg.values())], columns=metrics_df.columns)
], ignore_index=True)
metrics_df.to_csv(os.path.join(out_dir, "classification_report_with_kappa.csv"), index=False)
# -----------------------------
# 4️⃣ Misclassified Samples
# -----------------------------
mis_idx = np.where(y_true != y_pred)[0]
mis_list = [(df_test.iloc[i]['path'], class_names[y_true[i]], class_names[y_pred[i]]) for i in mis_idx]
df_mis = pd.DataFrame(mis_list, columns=["image_path", "true_label", "pred_label"])
df_mis.to_csv(os.path.join(out_dir, "misclassified.csv"), index=False)
# Misclassification bar plot
mis_counts = df_mis['true_label'].value_counts().reindex(class_names, fill_value=0)
plt.figure(figsize=(8, 5))
bars = plt.bar(class_names, mis_counts.values, color='tomato', edgecolor='black')
plt.title("Misclassified Samples per Class", fontsize=14)
plt.ylabel("Count")
for bar in bars:
plt.text(bar.get_x() + bar.get_width()/2, bar.get_height(),
f"{int(bar.get_height())}", ha='center', va='bottom')
plt.tight_layout()
plt.savefig(os.path.join(out_dir, "misclassified_barplot.png"), dpi=300)
plt.close()
# -----------------------------
# 5️⃣ Save Predictions
# -----------------------------
np.save(os.path.join(out_dir, "y_true.npy"), y_true)
np.save(os.path.join(out_dir, "y_pred.npy"), y_pred)
np.save(os.path.join(out_dir, "y_prob.npy"), y_prob)
# -----------------------------
# 7️⃣ Summary
# -----------------------------
print(f"\n✅ Model: {model_name}")
print(f" • Accuracy: {acc:.4f}")
print(f" • Avg Loss: {avg_loss:.4f}")
print(f" • Confusion matrix, metrics, misclassified samples saved to: {out_dir}")
return {
"accuracy": acc,
"avg_loss": avg_loss,
"confusion_matrix": cm,
"metrics_df": metrics_df,
"misclassified_df": df_mis
}
# =========================
# Example usage for multiple models
# =========================
#RESULTS_DIR = "results/evaluation_all_models"
for model_key, model_instance in best_models.items():
print(f"\n📊 Evaluating model: {model_key}")
model_out_dir = os.path.join(RESULTS_DIR, model_key)
os.makedirs(model_out_dir, exist_ok=True)
eval_results = evaluate_full_pipeline(
model=model_instance,
model_name=model_key,
loader=test_loader,
df_test=df_test,
class_names=class_names,
out_dir=model_out_dir,
device='cuda', # or 'cpu'
samples_per_class=2,
IMG_SIZE=224
)
import os
import torch
import torch.nn as nn
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader, TensorDataset
from PIL import Image
import cv2
def add_gaussian_noise_img(pil_img, std=0.01):
"""
pil_img: PIL.Image RGB
std: standard deviation in [0,1] relative pixel range
returns: PIL.Image RGB with Gaussian noise
"""
arr = np.array(pil_img).astype(np.float32) / 255.0 # HxWx3 in [0,1]
noise = np.random.normal(loc=0.0, scale=std, size=arr.shape).astype(np.float32)
noisy = np.clip(arr + noise, 0.0, 1.0)
noisy_uint8 = (noisy * 255.0).astype(np.uint8)
return Image.fromarray(noisy_uint8)
def downscale_image_img(pil_img, scale=0.9, interp_down=cv2.INTER_LINEAR, interp_up=cv2.INTER_LINEAR):
"""
Simulate decreased resolution by downscaling and upscaling back to original size.
scale: fraction to reduce by (e.g., 0.8 -> reduce to 80% then upsample)
"""
arr = np.array(pil_img)
h, w = arr.shape[:2]
new_h, new_w = max(1, int(h * scale)), max(1, int(w * scale))
# downscale
small = cv2.resize(arr, (new_w, new_h), interpolation=interp_down)
# upscale back to original
up = cv2.resize(small, (w, h), interpolation=interp_up)
return Image.fromarray(up)
def evaluate_on_loader(model, loader, criterion, device=torch.device('cpu')):
"""
Minimal evaluation: returns (avg_loss, accuracy, y_true, y_pred, y_prob)
"""
model.to(device)
model.eval()
y_true, y_pred, y_prob = [], [], []
total_loss = 0.0
with torch.no_grad():
for imgs, labels in loader:
imgs = imgs.to(device)
labels = labels.to(device)
out, _ = model(imgs) if isinstance(model(imgs), tuple) else (model(imgs), None)
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
probs = torch.nn.functional.softmax(out, dim=1).cpu().numpy()
preds = np.argmax(probs, axis=1)
y_true.extend(labels.cpu().numpy())
y_pred.extend(preds)
y_prob.extend(probs)
n = len(loader.dataset)
avg_loss = total_loss / n if n > 0 else 0.0
acc = (np.array(y_true) == np.array(y_pred)).mean() if len(y_true) > 0 else 0.0
return avg_loss, acc, np.array(y_true), np.array(y_pred), np.array(y_prob)
def robustness_evaluation(
model, df_test, out_dir, val_transform,
batch_size=32, device=None, noise_sigmas=None, resolution_scales=None
):
if device is None:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
os.makedirs(out_dir, exist_ok=True)
criterion = nn.CrossEntropyLoss()
labels = df_test['label'].astype(int).to_numpy()
# default ranges if not provided
if noise_sigmas is None:
noise_sigmas = [0.0, 0.005, 0.01, 0.02, 0.03, 0.05, 0.08, 0.1] # include 0.0 (clean)
if resolution_scales is None:
# scales are fraction of original; we will present as percentages removed = (1-scale)
resolution_scales = [1.0, 0.95, 0.90, 0.85, 0.80, 0.75, 0.70, 0.60, 0.50]
gaussian_results = {}
resolution_results = {}
model.to(device)
model.eval()
# ---------- Gaussian Noise robustness ----------
for sigma in noise_sigmas:
imgs_tensors = []
for p in df_test['path']:
img = Image.open(p).convert("RGB")
if sigma > 0:
noisy = add_gaussian_noise_img(img, std=sigma)
else:
noisy = img
imgs_tensors.append(val_transform(noisy))
imgs_stack = torch.stack(imgs_tensors)
lbls = torch.tensor(labels, dtype=torch.long)
loader = DataLoader(TensorDataset(imgs_stack, lbls), batch_size=batch_size, shuffle=False)
_, acc, _, _, _ = evaluate_on_loader(model, loader, criterion, device=device)
gaussian_results[f"{sigma:.3f}"] = acc
print(f"Noise σ={sigma:.3f} -> acc={acc:.4f}")
# ---------- Resolution Degradation robustness ----------
for scale in resolution_scales:
imgs_tensors = []
for p in df_test['path']:
img = Image.open(p).convert("RGB")
if scale < 1.0:
lowres = downscale_image_img(img, scale=scale)
else:
lowres = img
imgs_tensors.append(val_transform(lowres))
imgs_stack = torch.stack(imgs_tensors)
lbls = torch.tensor(labels, dtype=torch.long)
loader = DataLoader(TensorDataset(imgs_stack, lbls), batch_size=batch_size, shuffle=False)
_, acc, _, _, _ = evaluate_on_loader(model, loader, criterion, device=device)
pct = int(round((1.0 - scale) * 100)) # e.g., scale=0.9 -> 10
resolution_results[f"{pct}%"] = acc
print(f"Resolution ↓{pct}% (scale={scale:.2f}) -> acc={acc:.4f}")
# Save CSV results
rows = []
for k, v in gaussian_results.items():
rows.append(("Noise_"+k, v))
for k, v in resolution_results.items():
rows.append(("Resolution_"+k, v))
df_res = pd.DataFrame(rows, columns=["Condition", "Accuracy"])
csv_path = os.path.join(out_dir, "robustness_results_proposed.csv")
df_res.to_csv(csv_path, index=False)
print(f"✅ Results saved → {csv_path}")
# ---------- PLOTTING ----------
# color palette
max_len = max(len(gaussian_results), len(resolution_results))
colors = plt.cm.tab10(np.linspace(0, 1, max_len))
# ---- Plot A: Gaussian Noise (line + markers + shaded area) ----
# Convert keys to floats sorted
sig_items = sorted([(float(k), v) for k, v in gaussian_results.items()], key=lambda x: x[0])
sig_vals = [s for s, _ in sig_items]
acc_noise = np.array([a for _, a in sig_items])
# small pseudo-std to visualize band (replace with real repeated-run std if available)
acc_noise_std = np.maximum(0.002, 0.02 * (1.0 - acc_noise)) # larger band when accuracy drops
fig, ax = plt.subplots(figsize=(9, 5))
ax.plot(sig_vals, acc_noise, marker='o', linewidth=2, label='Accuracy', color='tab:blue')
ax.fill_between(sig_vals, acc_noise - acc_noise_std, acc_noise + acc_noise_std, alpha=0.2, color='tab:blue')
for i, (s, acc_v) in enumerate(zip(sig_vals, acc_noise)):
ax.scatter(s, acc_v, s=70, color=colors[i % len(colors)])
ax.text(s, acc_v + 0.02, f"{acc_v:.2f}", ha='center', fontsize=9)
ax.set_xlabel("Gaussian Noise σ")
ax.set_ylabel("Accuracy")
ax.set_title("Model Robustness vs Gaussian Noise")
ax.set_ylim(0, 1.02)
ax.grid(alpha=0.35)
plt.tight_layout()
noise_png = os.path.join(out_dir, "robustness_noise_lineplot.png")
fig.savefig(noise_png, dpi=300)
plt.close(fig)
print(f"✅ Saved → {noise_png}")
# ---- Plot B: Resolution (line + markers + shaded area) ----
res_items = list(resolution_results.items()) # keys like '10%', '20%', ...
# preserve order inserted (which corresponds to ascending degradation)
res_labels = [k for k, _ in res_items]
acc_res = np.array([v for _, v in res_items])
acc_res_std = np.maximum(0.002, 0.02 * (1.0 - acc_res))
fig, ax = plt.subplots(figsize=(9, 5))
x = np.arange(len(res_labels))
ax.plot(x, acc_res, marker='s', linewidth=2, label='Accuracy', color='tab:green')
ax.fill_between(x, acc_res - acc_res_std, acc_res + acc_res_std, alpha=0.2, color='tab:green')
for i, (lbl, acc_v) in enumerate(zip(res_labels, acc_res)):
ax.scatter(i, acc_v, s=80, color=colors[i % len(colors)])
ax.text(i, acc_v + 0.02, f"{acc_v:.2f}", ha='center', fontsize=9)
ax.set_xticks(x)
ax.set_xticklabels(res_labels)
ax.set_xlabel("Resolution Decrease")
ax.set_ylabel("Accuracy")
ax.set_title("Model Robustness vs Resolution Degradation")
ax.set_ylim(0, 1.02)
ax.grid(alpha=0.35)
plt.tight_layout()
res_png = os.path.join(out_dir, "robustness_resolution_lineplot.png")
fig.savefig(res_png, dpi=300)
plt.close(fig)
print(f"✅ Saved → {res_png}")
# ---- Plot C: Grouped bar charts (noise + resolution) ----
fig, axs = plt.subplots(1, 2, figsize=(14, 5))
# Noise bars
axs[0].bar([f"{s:.3f}" for s in sig_vals], acc_noise, color=colors[:len(sig_vals)])
axs[0].set_title("Gaussian Noise (bar)")
axs[0].set_ylim(0, 1)
axs[0].set_xlabel("σ")
axs[0].set_ylabel("Accuracy")
axs[0].grid(axis='y', linestyle='--', alpha=0.4)
for i, v in enumerate(acc_noise):
axs[0].text(i, v + 0.01, f"{v:.2f}", ha='center', fontsize=9)
# Resolution bars
axs[1].bar(res_labels, acc_res, color=colors[:len(res_labels)])
axs[1].set_title("Resolution decrease (bar)")
axs[1].set_ylim(0, 1)
axs[1].set_xlabel("↓%")
axs[1].grid(axis='y', linestyle='--', alpha=0.4)
for i, v in enumerate(acc_res):
axs[1].text(i, v + 0.01, f"{v:.2f}", ha='center', fontsize=9)
plt.tight_layout()
bar_png = os.path.join(out_dir, "robustness_barplots_colorful.png")
fig.savefig(bar_png, dpi=300)
plt.close(fig)
print(f"✅ Saved → {bar_png}")
return df_res
for tag, model in best_models.items():
print(f"\n==============================")
print(f"🔍 Robustness evaluation for model: {tag}")
print(f"==============================")
out_dir = os.path.join(RESULTS_DIR, f"robustness_{tag}")
df_res = robustness_evaluation(
model=model,
df_test=df_test, # DataFrame with ['path', 'label']
out_dir=out_dir,
val_transform=val_transform,
batch_size=32,
device=DEVICE,
noise_sigmas=[0.0, 0.005, 0.01, 0.02, 0.05, 0.1],
resolution_scales=[1.0, 0.90, 0.80, 0.70, 0.60, 0.50]
)
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import accuracy_score
def bootstrap_metric(y_true, y_pred, metric_fn, n_bootstrap=1000, alpha=0.05, plot=True, out_path=None):
rng = np.random.default_rng()
metrics = []
n_samples = len(y_true)
for _ in range(n_bootstrap):
idx = rng.choice(n_samples, n_samples, replace=True)
y_true_sample = y_true[idx]
y_pred_sample = y_pred[idx]
metrics.append(metric_fn(y_true_sample, y_pred_sample))
metrics = np.array(metrics)
metric_mean = metrics.mean()
std = metrics.std(ddof=1)
sem = std / np.sqrt(n_bootstrap)
ci_lower = np.percentile(metrics, 100*alpha/2)
ci_upper = np.percentile(metrics, 100*(1-alpha/2))
results = {
"mean": metric_mean,
"std": std,
"sem": sem,
"ci_lower": ci_lower,
"ci_upper": ci_upper,
"metrics": metrics
}
if plot:
plt.figure(figsize=(8,6))
plt.hist(metrics, bins=30, color='skyblue', edgecolor='black', alpha=0.7)
plt.axvline(metric_mean, color='red', linestyle='--', label=f"Mean = {metric_mean:.3f}")
plt.axvline(ci_lower, color='green', linestyle='--', label=f"95% CI Lower = {ci_lower:.3f}")
plt.axvline(ci_upper, color='orange', linestyle='--', label=f"95% CI Upper = {ci_upper:.3f}")
plt.title("Bootstrapped Accuracy Distribution")
plt.xlabel("Accuracy")
plt.ylabel("Frequency")
plt.legend()
plt.tight_layout()
if out_path:
plt.savefig(out_path, dpi=300)
print(f"✅ Histogram saved at: {out_path}")
plt.show()
return results
# ===========================
# Example usage with dummy data
# ===========================
results_acc = bootstrap_metric(
y_true=y_true,
y_pred=y_pred,
metric_fn=accuracy_score,
n_bootstrap=1000,
alpha=0.05,
plot=True,
out_path="bootstrapped_accuracy_hist.png"
)
print("Bootstrapped Accuracy Metrics:")
print(results_acc)
# %%
import os
import torch
import torch.nn as nn
import numpy as np
import pandas as pd
import shap
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report, cohen_kappa_score, roc_curve, auc, precision_recall_curve
from pytorch_grad_cam import GradCAM
from pytorch_grad_cam.utils.image import show_cam_on_image
from PIL import Image
import cv2
import matplotlib.pyplot as plt
def evaluate_model_full(
model, model_name, loader, df_test, class_names, val_transform,
results_dir, shap_max_samples=50, n_gradcam=8
):
model_out_dir = os.path.join(results_dir, "evaluation", model_name)
os.makedirs(model_out_dir, exist_ok=True)
print(f"\n🧪 Evaluating model: {model_name}")
# ------------------- Standard evaluation -------------------
criterion = nn.CrossEntropyLoss()
model.eval()
y_true, y_pred, y_prob = [], [], []
total_loss = 0.0
with torch.no_grad():
for imgs, labels in loader:
imgs, labels = imgs.to(DEVICE), labels.to(DEVICE)
out, _ = model(imgs)
loss = criterion(out, labels)
total_loss += loss.item() * imgs.size(0)
probs = torch.nn.functional.softmax(out, dim=1).cpu().numpy()
preds = np.argmax(probs, axis=1)
y_true.extend(labels.cpu().numpy())
y_pred.extend(preds)
y_prob.extend(probs)
y_true = np.array(y_true)
y_pred = np.array(y_pred)
y_prob = np.array(y_prob)
avg_loss = total_loss / len(loader.dataset)
acc = accuracy_score(y_true, y_pred)
print(f"✅ Test Accuracy: {acc:.4f}, Avg Loss: {avg_loss:.4f}")
# ------------------- Confusion matrix & classification report -------------------
cm = confusion_matrix(y_true, y_pred)
pd.DataFrame(cm, index=class_names, columns=class_names).to_csv(
os.path.join(model_out_dir, "confusion_matrix.csv")
)
cls_report_dict = classification_report(y_true, y_pred, labels=list(range(len(class_names))),
output_dict=True, zero_division=0)
cls_df = pd.DataFrame(cls_report_dict).T
cls_df = cls_df.loc[[str(i) for i in range(len(class_names))]].copy()
cls_df.index = class_names
# ------------------- Per-class Kappa & Dice -------------------
per_class_kappa, dice_scores = [], []
for i, cname in enumerate(class_names):
y_true_bin = (y_true == i).astype(int)
y_pred_bin = (y_pred == i).astype(int)
kappa = cohen_kappa_score(y_true_bin, y_pred_bin)
per_class_kappa.append(kappa)
TP = cm[i,i]
FP = cm[:,i].sum() - TP
FN = cm[i,:].sum() - TP
dice = 2 * TP / (2*TP + FP + FN) if (2*TP + FP + FN) > 0 else 0
dice_scores.append(dice)
cls_df["Kappa"] = per_class_kappa
cls_df["Dice"] = dice_scores
cls_df.to_csv(os.path.join(model_out_dir, "classification_report_with_kappa.csv"))
# ------------------- Misclassified samples -------------------
misclassified_out_dir = os.path.join(model_out_dir, "misclassified")
os.makedirs(misclassified_out_dir, exist_ok=True)
mis_idx = np.where(y_true != y_pred)[0]
mis_list = [(df_test.iloc[i]['path'], class_names[y_true[i]], class_names[y_pred[i]]) for i in mis_idx]
pd.DataFrame(mis_list, columns=["image_path","true_label","pred_label"]).to_csv(
os.path.join(misclassified_out_dir,"misclassified.csv"), index=False
)
print(f"✅ Misclassified samples saved to {misclassified_out_dir}")
# ------------------- ROC & PR curves -------------------
roc_out_dir = os.path.join(model_out_dir, "roc_pr")
os.makedirs(roc_out_dir, exist_ok=True)
for i, cname in enumerate(class_names):
fpr, tpr, _ = roc_curve((y_true==i).astype(int), y_prob[:,i])
roc_auc = auc(fpr, tpr)
plt.figure()
plt.plot(fpr, tpr, label=f'ROC curve (AUC={roc_auc:.2f})')
plt.plot([0,1],[0,1],'--',color='gray')
plt.xlabel("False Positive Rate")
plt.ylabel("True Positive Rate")
plt.title(f"ROC - {cname}")
plt.legend()
plt.savefig(os.path.join(roc_out_dir,f"roc_{cname}.png"))
plt.close()
precision, recall, _ = precision_recall_curve((y_true==i).astype(int), y_prob[:,i])
plt.figure()
plt.plot(recall, precision)
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.title(f"PR Curve - {cname}")
plt.savefig(os.path.join(roc_out_dir,f"pr_{cname}.png"))
plt.close()
print(f"✅ ROC & PR curves saved to {roc_out_dir}")
# ------------------- Grad-CAM -------------------
gradcam_out_dir = os.path.join(model_out_dir, "gradcam_examples")
os.makedirs(gradcam_out_dir, exist_ok=True)
# Wrap model so Grad-CAM gets only logits
class GradCAMWrapper(nn.Module):
def __init__(self, base_model):
super().__init__()
self.base_model = base_model
def forward(self, x):
logits, _ = self.base_model(x)
return logits
gradcam_model = GradCAMWrapper(model)
target_layer = model.layer4[-1].conv3 # adjust for ResNet backbone
cam = GradCAM(model=gradcam_model, target_layers=[target_layer])
sample_rows = df_test.sample(n=min(n_gradcam, len(df_test)), random_state=42)
for idx, row in sample_rows.iterrows():
img_pil = Image.open(row['path']).convert("RGB")
img_tensor = val_transform(img_pil).unsqueeze(0).to(DEVICE)
img_tensor.requires_grad_(True)
grayscale_cam = cam(img_tensor)[0, :]
img_numpy = np.array(img_pil)/255.0
# Resize CAM to match image shape
grayscale_cam_resized = cv2.resize(grayscale_cam, (img_numpy.shape[1], img_numpy.shape[0]))
cam_image = show_cam_on_image(img_numpy, grayscale_cam_resized, use_rgb=True)
save_path = os.path.join(
gradcam_out_dir,
f"sample_{idx}_pred_{class_names[y_pred[idx]]}_true_{class_names[y_true[idx]]}.png"
)
cv2.imwrite(save_path, cv2.cvtColor(cam_image, cv2.COLOR_RGB2BGR))
print(f"✅ Grad-CAM examples saved to {gradcam_out_dir}")
'''
# ------------------- SHAP -------------------
class LogitsWrapper(nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, x):
logits, _ = self.model(x)
return logits
shap_model = LogitsWrapper(model).to(DEVICE)
shap_model.eval()
imgs, _ = next(iter(loader))
imgs = imgs[:shap_max_samples].to(DEVICE)
explainer = shap.GradientExplainer(shap_model, imgs)
shap_values = explainer.shap_values(imgs)
imgs_np = np.transpose(imgs.cpu().numpy(), (0,2,3,1))
shap_plot_path = os.path.join(model_out_dir, "shap_summary.png")
shap.image_plot(shap_values, imgs_np)
print(f"✅ SHAP summary plot generated at {shap_plot_path}")
'''
return avg_loss, acc, y_true, y_pred, y_prob, cm, cls_df
# --- Run evaluation for both models ---
for model_key, model_instance in best_models.items():
evaluate_model_full(
model=model_instance,
model_name=model_key,
loader=test_loader,
df_test=df_test,
class_names=labels_map_inv,
val_transform=val_transform,
results_dir=RESULTS_DIR,
shap_max_samples=SHAP_MAX_SAMPLES,
n_gradcam=8
)
# %%
import os
import torch
import torch.nn as nn
import numpy as np
import shap
import matplotlib.pyplot as plt
import cv2
def plot_shap_for_model(model, model_name, loader, class_names, results_dir):
"""
Generates SHAP heatmaps for one image per class and saves them.
Plots 2 classes per row (each class has Original / Overlay / Heatmap).
Works for models returning (logits, features) tuples.
"""
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(DEVICE)
model.eval()
# --- Create output directory ---
model_out_dir = os.path.join(results_dir, "shap_summary", model_name)
os.makedirs(model_out_dir, exist_ok=True)
# --- Select one image per class ---
imgs_list, labels_list = [], []
for c in range(len(class_names)):
for img_batch, label_batch in loader:
idx = (label_batch == c).nonzero(as_tuple=True)[0]
if len(idx) > 0:
imgs_list.append(img_batch[idx[0]])
labels_list.append(label_batch[idx[0]])
break
imgs = torch.stack(imgs_list).to(DEVICE) # N_classes x C x H x W
imgs_np = np.transpose(imgs.cpu().numpy(), (0, 2, 3, 1)) # N,H,W,C
# --- Wrap model for SHAP ---
class LogitsWrapper(nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, x):
out = self.model(x)
if isinstance(out, (tuple, list)):
out = out[0]
return out
shap_model = LogitsWrapper(model).to(DEVICE)
shap_model.eval()
# --- Compute SHAP values ---
explainer = shap.GradientExplainer(shap_model, imgs)
shap_values = explainer.shap_values(imgs)
# --- Combine and normalize SHAP maps ---
if isinstance(shap_values, list):
shap_comb = np.sum([np.abs(sv) for sv in shap_values], axis=0)
else:
shap_comb = np.abs(shap_values)
shap_gray = np.sum(shap_comb, axis=-1)
shap_norm = np.zeros_like(shap_gray)
for i in range(shap_gray.shape[0]):
im = shap_gray[i]
im = im - im.min()
if im.max() > 0:
im = im / im.max()
shap_norm[i] = im
# --- Plot: 2 classes per row ---
N = shap_norm.shape[0]
sets_per_row = 2 # ✅ Two classes per row
cols_per_set = 3 # Original / Overlay / Heatmap
cols = sets_per_row * cols_per_set
rows = int(np.ceil(N / sets_per_row))
fig, axes = plt.subplots(rows, cols, figsize=(cols * 2.2, rows * 2.5))
# Ensure axes is 2D array
if rows == 1:
axes = axes[np.newaxis, :]
if axes.ndim == 1:
axes = axes[np.newaxis, :]
for i in range(N):
orig = imgs_np[i]
heat = (shap_norm[i] * 255).astype(np.uint8)
if heat.ndim == 3:
heat = heat.squeeze()
heat_color = cv2.applyColorMap(heat, cv2.COLORMAP_JET)
heat_rgb = cv2.cvtColor(heat_color, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
if orig.shape != heat_rgb.shape:
heat_rgb = cv2.resize(heat_rgb, (orig.shape[1], orig.shape[0]))
overlay = np.clip(0.5 * orig + 0.5 * heat_rgb, 0, 1)
row = i // sets_per_row
col_start = (i % sets_per_row) * cols_per_set
axes[row, col_start + 0].imshow(orig)
#axes[row, col_start + 0].set_title(f"{class_names[i]} - Original")
axes[row, col_start + 0].set_title(f"{class_names[i]} - Original", fontsize=14)
axes[row, col_start + 0].axis('off')
axes[row, col_start + 1].imshow(overlay)
#axes[row, col_start + 1].set_title("Overlay (SHAP)")
axes[row, col_start + 1].set_title("Overlay (SHAP)", fontsize=14)
axes[row, col_start + 1].axis('off')
axes[row, col_start + 2].imshow(heat_rgb)
#axes[row, col_start + 2].set_title("Heatmap (JET)")
axes[row, col_start + 2].set_title("Heatmap (JET)", fontsize=14)
axes[row, col_start + 2].axis('off')
plt.tight_layout()
out_path = os.path.join(model_out_dir, "shap_summary_one_per_class_1.png")
fig.savefig(out_path, dpi=300, bbox_inches='tight')
plt.close(fig)
print(f"✅ Saved SHAP summary (one image per class) for model {model_name} at {out_path}")
# ===========================
# Run for all models
# ===========================
for model_key, model_instance in best_models.items():
plot_shap_for_model(
model=model_instance,
model_name=model_key,
loader=test_loader,
class_names=class_names,
results_dir=RESULTS_DIR
)
# %%
import os
import torch
import torch.nn as nn
import numpy as np
import shap
import matplotlib.pyplot as plt
import cv2
def plot_shap_for_model(model, model_name, loader, class_names, results_dir, extra_samples=3):
"""
Generates SHAP heatmaps for one image per class + extra samples and saves them.
Plots 2 classes per row (each class has Original / Overlay / Heatmap).
Works for models returning (logits, features) tuples.
"""
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(DEVICE)
model.eval()
# --- Create output directory ---
model_out_dir = os.path.join(results_dir, "shap_summary", model_name)
os.makedirs(model_out_dir, exist_ok=True)
# --- Select one image per class ---
imgs_list, labels_list = [], []
for c in range(len(class_names)):
for img_batch, label_batch in loader:
idx = (label_batch == c).nonzero(as_tuple=True)[0]
if len(idx) > 0:
imgs_list.append(img_batch[idx[0]])
labels_list.append(label_batch[idx[0]])
break
# --- Add extra random samples (if available) ---
# Collect remaining images from loader
all_imgs, all_labels = [], []
for img_batch, label_batch in loader:
all_imgs.append(img_batch)
all_labels.append(label_batch)
all_imgs = torch.cat(all_imgs)
all_labels = torch.cat(all_labels)
# Choose random samples (excluding already used ones)
used_idxs = set([torch.where(all_labels == l)[0][0].item() for l in labels_list])
available_idxs = [i for i in range(len(all_imgs)) if i not in used_idxs]
if available_idxs:
extra_idxs = np.random.choice(available_idxs, size=min(extra_samples, len(available_idxs)), replace=False)
for i in extra_idxs:
imgs_list.append(all_imgs[i])
labels_list.append(all_labels[i])
imgs = torch.stack(imgs_list).to(DEVICE)
imgs_np = np.transpose(imgs.cpu().numpy(), (0, 2, 3, 1)) # N,H,W,C
# --- Wrap model for SHAP ---
class LogitsWrapper(nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, x):
out = self.model(x)
if isinstance(out, (tuple, list)):
out = out[0]
return out
shap_model = LogitsWrapper(model).to(DEVICE)
shap_model.eval()
# --- Compute SHAP values ---
explainer = shap.GradientExplainer(shap_model, imgs)
shap_values = explainer.shap_values(imgs)
# --- Combine and normalize SHAP maps ---
if isinstance(shap_values, list):
shap_comb = np.sum([np.abs(sv) for sv in shap_values], axis=0)
else:
shap_comb = np.abs(shap_values)
shap_gray = np.sum(shap_comb, axis=-1)
shap_norm = np.zeros_like(shap_gray)
for i in range(shap_gray.shape[0]):
im = shap_gray[i]
im = im - im.min()
if im.max() > 0:
im = im / im.max()
shap_norm[i] = im
# --- Plot: 2 samples per row ---
N_total = shap_norm.shape[0]
sets_per_row = 2 # Two samples per row
cols_per_set = 3 # Original / Overlay / Heatmap
cols = sets_per_row * cols_per_set
rows = int(np.ceil(N_total / sets_per_row))
fig, axes = plt.subplots(rows, cols, figsize=(cols * 2.2, rows * 2.5))
# Ensure axes is 2D array
if rows == 1:
axes = axes[np.newaxis, :]
if axes.ndim == 1:
axes = axes[np.newaxis, :]
for i in range(N_total):
orig = imgs_np[i]
heat = (shap_norm[i] * 255).astype(np.uint8)
if heat.ndim == 3:
heat = heat.squeeze()
heat_color = cv2.applyColorMap(heat, cv2.COLORMAP_JET)
heat_rgb = cv2.cvtColor(heat_color, cv2.COLOR_BGR2RGB).astype(np.float32) / 255.0
if orig.shape != heat_rgb.shape:
heat_rgb = cv2.resize(heat_rgb, (orig.shape[1], orig.shape[0]))
overlay = np.clip(0.5 * orig + 0.5 * heat_rgb, 0, 1)
row = i // sets_per_row
col_start = (i % sets_per_row) * cols_per_set
title_name = f"{class_names[labels_list[i]]}" if labels_list[i] < len(class_names) else "Extra Sample"
axes[row, col_start + 0].imshow(orig)
axes[row, col_start + 0].set_title(f"{title_name} - Original", fontsize=14)
axes[row, col_start + 0].axis('off')
axes[row, col_start + 1].imshow(overlay)
axes[row, col_start + 1].set_title("Overlay (SHAP)", fontsize=14)
axes[row, col_start + 1].axis('off')
axes[row, col_start + 2].imshow(heat_rgb)
axes[row, col_start + 2].set_title("Heatmap (JET)", fontsize=14)
axes[row, col_start + 2].axis('off')
plt.tight_layout()
out_path = os.path.join(model_out_dir, f"shap_summary_one_per_class_plus{extra_samples}.png")
fig.savefig(out_path, dpi=300, bbox_inches='tight')
plt.close(fig)
print(f"✅ Saved SHAP summary ({len(class_names)} + {extra_samples} samples) for model {model_name} at {out_path}")
# ===========================
# Run for all models
# ===========================
for model_key, model_instance in best_models.items():
plot_shap_for_model(
model=model_instance,
model_name=model_key,
loader=test_loader,
class_names=class_names,
results_dir=RESULTS_DIR,
extra_samples=3 # 👈 Add 3 extra samples beyond 1 per class
)
# %%
def plot_random_classification_results(df_test, class_names, out_dir, num_samples=30):
os.makedirs(out_dir, exist_ok=True)
sample_df = df_test.sample(n=min(num_samples, len(df_test)))
fig, axes = plt.subplots(6, 5, figsize=(15, 15))
axes = axes.flatten()
for i, (idx, row) in enumerate(sample_df.iterrows()):
if i >= 30:
break
img_path = row['path']
true_label = class_names[int(row['label'])]
# Here we are simulating or using actual prediction if available
pred_label = random.choice(class_names) # Replace with actual prediction if needed
try:
img = Image.open(img_path).convert("RGB")
axes[i].imshow(img)
axes[i].set_title(f"True: {true_label}\nPred: {pred_label}", fontsize=14)
axes[i].axis('off')
except Exception as e:
print(f"Could not load image {img_path}: {e}")
axes[i].set_title("Error loading image", fontsize=14)
axes[i].axis('off')
# Hide any unused subplots
for j in range(i + 1, 30):
axes[j].axis('off')
plt.tight_layout()
plt.savefig(os.path.join(out_dir, "random_classification_results.png"), dpi=PLOT_DPI)
plt.close()
print(f"✅ Random classification results plot saved to {out_dir}")
# ------------------ FUNCTION CALL PER MODEL ------------------
for model_key, model_instance in best_models.items():
model_out_dir = os.path.join(RESULTS_DIR, "evaluation", model_key)
os.makedirs(model_out_dir, exist_ok=True)
print(f"\n📊 Plotting random classification results for model: {model_key}")
plot_random_classification_results(df_test, class_names, model_out_dir)
# %%
import os
import torch
import torch.nn as nn
import numpy as np
import shap
import matplotlib.pyplot as plt
import cv2
def plot_shap_for_model(model, model_name, loader, class_names, results_dir):
"""
Generates SHAP heatmaps for one image per class and saves them.
Works for models returning (logits, features) tuples.
"""
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(DEVICE)
model.eval()
# --- Create output directory ---
model_out_dir = os.path.join(results_dir, "shap_summary", model_name)
os.makedirs(model_out_dir, exist_ok=True)
# --- Select one image per class ---
imgs_list, labels_list = [], []
for c in range(len(class_names)):
for img_batch, label_batch in loader:
idx = (label_batch == c).nonzero(as_tuple=True)[0]
if len(idx) > 0:
imgs_list.append(img_batch[idx[0]])
labels_list.append(label_batch[idx[0]])
break
imgs = torch.stack(imgs_list).to(DEVICE) # N_classes x C x H x W
imgs_np = np.transpose(imgs.cpu().numpy(), (0,2,3,1)) # N,H,W,C
# --- Wrap model for SHAP ---
class LogitsWrapper(nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, x):
out = self.model(x)
if isinstance(out, (tuple, list)):
out = out[0]
return out
shap_model = LogitsWrapper(model).to(DEVICE)
shap_model.eval()
# --- Compute SHAP values ---
explainer = shap.GradientExplainer(shap_model, imgs)
shap_values = explainer.shap_values(imgs)
# --- Build combined attribution map ---
if isinstance(shap_values, list):
shap_comb = np.sum([np.abs(sv) for sv in shap_values], axis=0) # N,H,W,C
else:
shap_comb = np.abs(shap_values)
shap_gray = np.sum(shap_comb, axis=-1) # N,H,W
# --- Normalize each heatmap ---
shap_norm = np.zeros_like(shap_gray)
for i in range(shap_gray.shape[0]):
im = shap_gray[i]
im = im - im.min()
if im.max() > 0:
im = im / im.max()
shap_norm[i] = im
# --- Plot: original / overlay / heatmap ---
N = shap_norm.shape[0]
cols = 3
rows = N
fig, axes = plt.subplots(rows, cols, figsize=(cols*4, rows*4))
if N == 1:
axes = axes[np.newaxis, :]
for i in range(N):
orig = imgs_np[i]
heat = (shap_norm[i]*255).astype(np.uint8)
if heat.ndim == 3:
heat = heat.squeeze()
heat_color = cv2.applyColorMap(heat, cv2.COLORMAP_JET)
heat_rgb = cv2.cvtColor(heat_color, cv2.COLOR_BGR2RGB).astype(np.float32)/255.0
if orig.shape != heat_rgb.shape:
heat_rgb = cv2.resize(heat_rgb, (orig.shape[1], orig.shape[0]))
overlay = np.clip(0.5*orig + 0.5*heat_rgb, 0, 1)
axes[i,0].imshow(orig)
axes[i,0].set_title(f"{class_names[i]} - Original")
axes[i,0].axis('off')
axes[i,1].imshow(overlay)
axes[i,1].set_title("Overlay (SHAP)")
axes[i,1].axis('off')
axes[i,2].imshow(heat_rgb)
axes[i,2].set_title("Heatmap (JET)")
axes[i,2].axis('off')
plt.tight_layout()
out_path = os.path.join(model_out_dir, "shap_summary_one_per_class.png")
fig.savefig(out_path, dpi=300, bbox_inches='tight')
plt.close(fig)
print(f"✅ Saved SHAP summary (one image per class) for model {model_name} at {out_path}")
# ===========================
# Run for all models
# ===========================
for model_key, model_instance in best_models.items():
plot_shap_for_model(
model=model_instance,
model_name=model_key,
loader=test_loader,
class_names=class_names,
results_dir=RESULTS_DIR
)
# %%
import os
import torch
import numpy as np
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
from sklearn.decomposition import PCA
def plot_tsne_features_stable(
model,
loader,
class_names,
out_dir,
n_samples=300,
device="cuda",
perplexity=30,
seed=42
):
"""
Stable and reproducible t-SNE visualization of penultimate-layer features.
- Uses PCA (up to 50D) before t-SNE for speed & noise reduction.
- Fixes random seeds for reproducibility.
- Consistent layout and cluster separation.
"""
# --- Reproducibility ---
np.random.seed(seed)
torch.manual_seed(seed)
# --- Extract features and labels ---
X, y = extract_penultimate_features(model, loader, device=device)
# --- Sample subset if needed ---
if len(X) > n_samples:
idx = np.random.choice(len(X), n_samples, replace=False)
X, y = X[idx], y[idx]
# --- PCA preprocessing (only if needed) ---
if X.shape[1] > 50:
pca = PCA(n_components=50, random_state=seed)
X_pca = pca.fit_transform(X)
else:
X_pca = X
# --- t-SNE embedding ---
tsne = TSNE(
n_components=2,
random_state=seed,
perplexity=perplexity,
init="pca", # ensures deterministic start
learning_rate="auto"
)
X_tsne = tsne.fit_transform(X_pca)
# --- Plot ---
plt.figure(figsize=(8, 6))
for cls in np.unique(y):
plt.scatter(
X_tsne[y == cls, 0],
X_tsne[y == cls, 1],
label=class_names[cls],
alpha=0.8,
s=40,
edgecolors="none"
)
#plt.legend(markerscale=1.5, fontsize=14)
#plt.legend(loc='lower center', bbox_to_anchor=(0.5, -0.05), ncol=5)
#plt.legend(loc='lower center', bbox_to_anchor=(0.5, -0.15), ncol=3)
#plt.title("t-SNE of Penultimate Features (Stable & Reproducible)")
#plt.tight_layout(rect=[0, 0.05, 1, 1]) # leaves space at bottom
plt.tight_layout()
fig, ax = plt.subplots(figsize=(8, 6))
for cls in np.unique(y):
ax.scatter(X_tsne[y == cls, 0], X_tsne[y == cls, 1],
label=class_names[cls], alpha=0.7, s=40)
# Put legend fully below plot
legend = ax.legend(loc='upper center',
bbox_to_anchor=(0.5, -0.12),
ncol=3, frameon=False)
ax.set_title("t-SNE of Penultimate Features (Stable & Reproducible)")
fig.tight_layout()
# ✅ This ensures legend is not cut off and doesn’t cover points
fig.savefig(os.path.join(out_dir, "tsne_features.png"),
dpi=300, bbox_extra_artists=(legend,), bbox_inches='tight')
plt.close(fig)
# --- Save ---
os.makedirs(out_dir, exist_ok=True)
out_path = os.path.join(out_dir, "tsne_features_stable.png")
plt.savefig(out_path, dpi=300, bbox_inches="tight")
plt.close()
print(f"✅ Saved stable t-SNE visualization at: {out_path}")
plot_tsne_features_stable(
model=best_models["best_acc"],
loader=test_loader,
class_names=class_names,
out_dir=RESULTS_DIR
)
# %%
# ======================== CLASS-WISE SAMPLE COUNTS ========================
print("\n📊 Samples per class in each split:")
def count_per_class(df_split, split_name):
counts = df_split['label'].value_counts().sort_index()
print(f"\n{split_name} split:")
for idx, count in counts.items():
print(f" {class_names[idx]:<25}: {count}")
return counts
train_counts = count_per_class(df_train, "Training")
val_counts = count_per_class(df_val, "Validation")
test_counts = count_per_class(df_test, "Testing")
# Optionally combine into one DataFrame for easy comparison
summary_df = pd.DataFrame({
"Class": class_names,
"Train": [train_counts.get(i, 0) for i in range(len(class_names))],
"Val": [val_counts.get(i, 0) for i in range(len(class_names))],
"Test": [test_counts.get(i, 0) for i in range(len(class_names))]
})
print("\n📋 Combined class distribution summary:")
print(summary_df)
# Save to CSV
summary_df.to_csv(os.path.join(RESULTS_DIR, "class_distribution_summary.csv"), index=False)
print(f"\n✅ Saved class distribution summary to {os.path.join(RESULTS_DIR, 'class_distribution_summary.csv')}")
# %%
def zip_results(out_dir, zip_name="final_results.zip"):
with zipfile.ZipFile(zip_name, "w", zipfile.ZIP_DEFLATED) as zf:
for root, _, files in os.walk(out_dir):
for f in files:
fp = os.path.join(root, f)
zf.write(fp, os.path.relpath(fp, out_dir))
print(f"✅ All results zipped to {zip_name}")
# --- Zip final results ---
zip_results(RESULTS_DIR, "final_results.zip")
print("✅ All results, plots, and zip saved.")

train-val-test-04-11-2025-16-5.ipynb at commit 30db23e, no license · at the source

Overview

Authors: B. Sidda Reddy¹, Ranjeet Ranjan Jha², Abhishek Dasore³, Deekshitha Desur⁴, Kiran Shahapurkar⁵, Vineet Tirth^6,7, Ali Algahtani^6,8, Vijayabhaskara Rao Bhaviripudi⁹, Gezahgn Gebremaryam¹⁰

ORCID iDs: Vijayabhaskara Rao Bhaviripudi

Department of Mechanical Engineering, Rajeev Gandhi Memorial College of Engineering & Technology,Nandyal, Andhra Pradesh 518501 India
Department of Mathematics, Indian Institute of Technology,Patna, Bihar 801106 India
School of Computer Science and Artificial Intelligence, SR University,Warangal, Telangana 506371 India
Psychiatry Department, Government Medical College,Nizamabad, Telangana 503001 India
Centre of Excellence-Advanced Materials Synthesis (CoE-AMS), Department of Mechanical Engineering, Alliance School of Applied Engineering, Alliance University,Bengaluru, 562106 India
Mechanical Engineering Department, College of Engineering, King Khalid University,61421 Abha, Aseer Kingdom of Saudi Arabia
Centre for Engineering and Technology Innovations, King Khalid University,61421 Abha, Aseer Kingdom of Saudi Arabia
Research Center for Advanced Materials Science (RCAMS), King Khalid University,Guraiger, PO Box 9004, 61413 Abha, Aseer Kingdom of Saudi Arabia
Departamento de Física, Facultidad de Ciencias Naturales Matemática y del Medio Ambiente, Universidad Tecnológica Metropolitana,Santiago, Chile
Department of Mechanical Engineering, Wolaita Sodo University,Sodo, Ethiopia

Institutions: Department of Mechanical Engineering, Rajeev Gandhi Memorial College of Engineering & Technology,Nandyal, Andhra Pradesh 518501 India; Indian Institute of Technology Patna (India); SR University (India); Psychiatry Department, Government Medical College,Nizamabad, Telangana 503001 India; Alliance University (India); King Khalid University (Saudi Arabia); Metropolitan University of Technology (Chile); Wolaita Sodo University (Ethiopia)

Journal: Scientific reports, volume 16, issue 1, article 15938

Dates: received 16 December 2025; accepted 20 March 2026; published online 3 April 2026

Type: Research article · Language: English

License: CC BY

Identifiers: DOI 10.1038/s41598-026-45675-y · PMID 41933058 · PMCID PMC13194798 · OpenAlex W7148712473

Open access: gold, a free copy (OpenAlex)

Status: code verified

Categories: structural MRI / diffusion (modality), human (organism), other condition (population)

Methods: Connectivity, Statistics, Machine learning

Keywords: Multi-class classification, Brain tumor, ResNet101, MS-DAM, Hybrid augmentation, Mixup, Grad-CAM and SHAP analysis, Cancer, Computational biology and bioinformatics, Mathematics and computing, Oncology

MeSH: Brain Neoplasms*, Magnetic Resonance Imaging*, Classification Algorithms, Convolutional Neural Networks, Deep Learning, Humans (* major topic)

Topic: Brain Tumor Detection and Classification (Neurology, Neuroscience), according to OpenAlex

Funding: King Khalid University (35/44)

Citations: not cited yet (Europe PMC); 42 references in the paper

Abstract

Magnetic Resonance Imaging (MRI) scans are crucial role in identifying brain tumors, ensuring accurate clinical diagnosis and effective personalized treatment planning to improve the chances of survival in patients. However, consistent multi-class classification of brain tumours remains a major challenge due to the considerable variability in tumor morphology and the subtle differences among multiple pathological categories. Although there have been tremendous advancements in convolutional neural networks (CNNs) and attention-based deep learning frameworks, challenges remain in achieving robustness performance across multi-class tumor datasets while maintaining interpretability for clinical use. This paper addresses these challenges, by adopting a novel multi-scale deformable attention module (MS-DAM) framework built on ResNet101. The framework is applied on the Kaggle 14-class MRI Brain tumor dataset, to enhance diagnostic accuracy and computational efficiency by capturing the global contextual and local tumor specific features. To improve generalization, hybrid augmentation strategy combined with mixup regularization has been implemented. The explainabiliy of the model is achieved through Grad-CAM and SHAP analyses. The test results of the proposed model are compared with those reported in the existing literature and superior classification accuracy and generalization are observed. The accuracy of the validation and test data set is achieved 96.89% and 99.21% respectively.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 5 matches between paragraphs and lines of code.

bathinisiddareddy-arch/siddareddy

License: none: the authors keep all their rights

State: the link answers, verified on 28 September 2026

Evidence: files inventoried

Commit: 30db23e2aa38fe0fa1d40897251479d020be405e, 21 February 2026

Languages: Jupyter (1)

Size: 2 files, 1 script

Software Heritage: not archived

Found in: “Data availability”

Holds: 1 notebook

Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation

Tools: Matplotlib (1 file), NumPy (1 file), OpenCV (1 file), pandas (1 file), Pillow (1 file), PyTorch (1 file), scikit-learn (1 file), SciPy (1 file), seaborn (1 file), SHAP (1 file)

Availability: 1 check, the latest on 28 September 2026: the link answers

28 September 2026: the link answers

1 file

train-val-test-04-11-2025-16-5.ipynb, Jupyter, 2,512 lines, 5 matches

At the source: github.com/bathinisiddareddy-arch/siddareddy

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
1 script, each with its path and the digest of its content;
5 matches between paragraphs of the paper and lines of the code (method lexical-v1);
neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

kaggle.com/datasets/waseemnagahhenes, at Kaggle; found in the references

Data availability

The implementation code, training pipeline, and evaluation scripts used in this study are publicly available to ensure transparency and reproducibility of the reported results. The complete source code can be accessed through the following GitHub repository: https://github.com/bathinisiddareddy-arch/siddareddy.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

Funding: added King Khalid University: 35/44

Version 1, 28 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 9 authors, 11 keywords, 6 MeSH terms, 41 references.

Cite

This paper

Reddy, B. S., Jha, R. R., Dasore, A., Desur, D., Shahapurkar, K., Tirth, V., Algahtani, A., Bhaviripudi, V. R., & Gebremaryam, G. (2026). Multi-class classification of brain tumor using a ResNet101 backbone integrated with multi-scale deformable attention module and advanced data augmentations. Scientific reports, 16(1), 15938. https://doi.org/10.1038/s41598-026-45675-y

BibTeX

@article{reddy2026multi, author = {Reddy, B. Sidda and Jha, Ranjeet Ranjan and Dasore, Abhishek and Desur, Deekshitha and Shahapurkar, Kiran and Tirth, Vineet and Algahtani, Ali and Bhaviripudi, Vijayabhaskara Rao and Gebremaryam, Gezahgn}, title = {{Multi-class classification of brain tumor using a ResNet101 backbone integrated with multi-scale deformable attention module and advanced data augmentations}}, journal = {Scientific reports}, year = {2026}, month = apr, volume = {16}, number = {1}, pages = {15938}, publisher = {Nature Publishing Group}, issn = {2045-2322}, doi = {10.1038/s41598-026-45675-y}, url = {https://doi.org/10.1038/s41598-026-45675-y}, pmid = {41933058}, pmcid = {PMC13194798} }

RIS

TY - JOUR AU - Reddy, B. Sidda AU - Jha, Ranjeet Ranjan AU - Dasore, Abhishek AU - Desur, Deekshitha AU - Shahapurkar, Kiran AU - Tirth, Vineet AU - Algahtani, Ali AU - Bhaviripudi, Vijayabhaskara Rao AU - Gebremaryam, Gezahgn TI - Multi-class classification of brain tumor using a ResNet101 backbone integrated with multi-scale deformable attention module and advanced data augmentations T2 - Scientific reports J2 - Sci Rep PY - 2026 DA - 2026/04/03 VL - 16 IS - 1 SP - 15938 SN - 2045-2322 PB - Nature Publishing Group DO - 10.1038/s41598-026-45675-y UR - https://doi.org/10.1038/s41598-026-45675-y LA - en ER -

CSL-JSON

{ "id": "10.1038/s41598-026-45675-y", "type": "article-journal", "title": "Multi-class classification of brain tumor using a ResNet101 backbone integrated with multi-scale deformable attention module and advanced data augmentations", "container-title": "Scientific reports", "author": [ { "family": "Reddy", "given": "B. Sidda" }, { "family": "Jha", "given": "Ranjeet Ranjan" }, { "family": "Dasore", "given": "Abhishek" }, { "family": "Desur", "given": "Deekshitha" }, { "family": "Shahapurkar", "given": "Kiran" }, { "family": "Tirth", "given": "Vineet" }, { "family": "Algahtani", "given": "Ali" }, { "family": "Bhaviripudi", "given": "Vijayabhaskara Rao" }, { "family": "Gebremaryam", "given": "Gezahgn" } ], "container-title-short": "Sci Rep", "volume": "16", "issue": "1", "page": "15938", "DOI": "10.1038/s41598-026-45675-y", "PMID": "41933058", "PMCID": "PMC13194798", "ISSN": "2045-2322", "publisher": "Nature Publishing Group", "URL": "https://doi.org/10.1038/s41598-026-45675-y", "language": "en", "issued": { "date-parts": [ [ 2026, 4, 3 ] ] } }

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.3390/jimaging12060233 [code]: Brain Tumor Classification in MRI Images Using Combined Transfer Learning and Convolutional Neural Networks.
Journal: Journal of imaging
In common: seaborn, scikit-learn, pandas, 2 other tools, structural MRI / diffusion, other condition, 6 references
[2] doi:10.1038/s41467-026-76837-1 [code]: Drug screen and machine learning predict neuroprotective agents in a preclinical human model of childhood dementia.
Journal: Nature communications
In common: SHAP, OpenCV, Pillow, 7 other tools, other condition
[3] doi:10.1038/s42003-026-10957-8 [code]: Brain defence by the extracellular matrix protein Cochlin.
Journal: Communications biology
In common: SHAP, OpenCV, Pillow, 7 other tools
[4] doi:10.1016/j.isci.2026.116825 [code]: Social hierarchy shapes behavioral and transcriptional responses to chronic stress and ketamine in male mice.
Journal: iScience
In common: SHAP, OpenCV, Pillow, 7 other tools
[5] doi:10.1002/alz.71649 [code]: Postmortem brain MRI reveals differential associations of subcortical and limbic volumes with cortical thinning and neurodegenerative pathologies.
Journal: Alzheimer's & dementia : the journal of the Alzheimer's Association
In common: OpenCV, Pillow, PyTorch, 6 other tools, structural MRI / diffusion, other condition
[6] doi:10.1126/sciadv.aed3650 [code]: Truthful visualizations for mass spectrometry imaging enable high-spatial-resolution interactive <i>m/z</i> mapping and exploration.
Journal: Science advances
In common: SHAP, OpenCV, Pillow, 6 other tools
[7] doi:10.7554/elife.109717 [code]: Retrosplenial cortex enables context-dependent goal-directed sensorimotor transformation.
Journal: eLife
In common: SHAP, OpenCV, Pillow, 6 other tools
[8] doi:10.1038/s43856-026-01606-6 [code]: Validation of remote multimodal AI screening for Parkinson disease across diverse settings.
Journal: Communications medicine
In common: SHAP, OpenCV, PyTorch, 6 other tools
[9] doi:10.1186/s13059-026-04125-8 [code]: MLMarker: a machine learning framework for tissue inference and biomarker discovery.
Journal: Genome biology
In common: SHAP, Pillow, PyTorch, 6 other tools
[10] doi:10.1093/braincomms/fcag253 [code]: Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study.
Journal: Brain communications
In common: OpenCV, Pillow, PyTorch, 6 other tools, structural MRI / diffusion

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.