OSCR

Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study.

Code ↔ Paper

4 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 4 matches
  1. [1] § Materials and methods › Artificial neural network › Model training and execution ↔ tutorials/Image_and_Text_Classification_LIME.ipynb, lines 400–451 · score 0.74 · cross entropy loss, Adam optimizer, trained models, epoch, classification, validation
  2. [2] § Materials and methods › Artificial neural network › Model training and execution ↔ tutorials/Titanic_Basic_Interpret.ipynb, lines 123–151 · score 0.58 · cross entropy loss, Adam, epoch, optimizer, trained
  3. [3] § Materials and methods › Artificial neural network › Architecture ↔ captum/attr/_core/layer/grad_cam.py, lines 26–58 · score 0.55 · convolutional neural network, convolution layer, channels, activation, dimensional
  4. [4] § Materials and methods › Artificial neural network › Architecture ↔ captum/attr/_core/guided_grad_cam.py, lines 23–53 · score 0.51 · convolutional neural network, convolution layer, dimensional

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 550 lines · 28 KB · BSD-3-Clause · 1 match

  1. # %% [markdown]
  2. # # LIME to Inspect Image & Text Classification
  4. # %% [markdown]
  5. # This tutorial focuses on showing how to use Captum's implementation of Local Interpretable Model-agnostic Explanations (LIME) to understand neural models. The following content is divided into an image classification section to present our high-level interface `Lime` class and a text classification section for the more customizable low-level interface `LimeBase`.
  7. # %%
  8. import torch
  9. import torch.nn.functional as F
  11. from captum.attr import visualization as viz
  12. from captum.attr import Lime, LimeBase
  13. from captum._utils.models.linear_model import SkLearnLinearRegression, SkLearnLasso
  15. import os
  16. import json
  18. # %% [markdown]
  19. # ## 1. Image Classification
  21. # %% [markdown]
  22. # In this section, we will learn applying Lime to analyze a Resnet trained on ImageNet-1k. For testing data, we use samples from PASCAL VOC 2012 since its segmentation masks can directly serve as semantic "super-pixels" for images.
  24. # %%
  25. from torchvision.models import resnet18
  26. from torchvision.datasets import VOCSegmentation
  27. import torchvision.transforms as T
  28. from captum.attr._core.lime import get_exp_kernel_similarity_function
  30. from PIL import Image
  31. import matplotlib.pyplot as plt
  33. # %% [markdown]
  34. # ### 1.1 Load the model and dataset
  36. # %% [markdown]
  37. # We can directly load the pretrained Resnet from torchvision and set it to evaluation mode as our target image classifier to inspect.
  39. # %%
  40. resnet = resnet18(pretrained=True)
  41. resnet = resnet.eval()
  43. # %% [markdown]
  44. # This model predicts ImageNet-1k labels for given sample images. To better present the results, we also load the mapping of label index and text.
  46. # %%
  47. !wget -P $HOME/.torch/models https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json
  49. # %%
  50. labels_path = os.getenv('HOME') + '/.torch/models/imagenet_class_index.json'
  51. with open(labels_path) as json_data:
  52. idx_to_labels = {idx: label for idx, [_, label] in json.load(json_data).items()}
  54. # %% [markdown]
  55. # As mentioned before, we will use PASCAL VOC 2012 as the test data, which is available in torchvision as well. We will load it with `torchvision` transforms which convert both the images and targets, i.e., segmentation masks, to tensors.
  57. # %%
  58. voc_ds = VOCSegmentation(
  59. './VOC',
  60. year='2012',
  61. image_set='train',
  62. download=False,
  63. transform=T.Compose([
  64. T.ToTensor(),
  65. T.Normalize(
  66. mean=[0.485, 0.456, 0.406],
  67. std=[0.229, 0.224, 0.225]
  68. )
  69. ]),
  70. target_transform=T.Lambda(
  71. lambda p: torch.tensor(p.getdata()).view(1, p.size[1], p.size[0])
  72. )
  73. )
  75. # %% [markdown]
  76. # This dataset provides an additional segmentation mask along with every image. Compared with inspecting each pixel, the segments (or "super-pixels") are semantically more intuitive for humans to perceive. We will discuss more in section 1.3.
  77. #
  78. # Let's pick one example to see how the image and corresponding mask look like. Here we choose an image with more than one segment besides the background, so that we can compare each segment's impact on the classification.
  80. # %%
  81. sample_idx = 439
  83. def show_image(ind):
  84. fig, ax = plt.subplots(1, 2, figsize=[6.4 * 2, 4.8])
  85. for i, (name, source) in enumerate(zip(['Image', 'Mask'], [voc_ds.images, voc_ds.masks])):
  86. ax[i].imshow(Image.open(source[ind]));
  87. ax[i].set_title(f"{name} {ind}")
  88. ax[i].axis('off')
  90. show_image(sample_idx)
  92. # %% [markdown]
  93. # ### 1.2 Baseline classification
  95. # %% [markdown]
  96. # We can check how well our model works with the above example. The original Resnet only gives the logits of labels, so we will add a softmax layer to normalize them into probabilities.
  98. # %%
  99. img, seg_mask = voc_ds[sample_idx] # tensors of shape (channel, hight, width)
  101. outputs = resnet(img.unsqueeze(0))
  102. output_probs = F.softmax(outputs, dim=1).squeeze(0)
  104. # %% [markdown]
  105. # Then we present the top 5 predicted labels to verify the result.
  107. # %%
  108. def print_result(probs, topk=1):
  109. probs, label_indices = torch.topk(probs, topk)
  110. probs = probs.tolist()
  111. label_indices = label_indices.tolist()
  112. for prob, idx in zip(probs, label_indices):
  113. label = idx_to_labels[str(idx)]
  114. print(f'{label} ({idx}):', round(prob, 4))
  116. print_result(output_probs, topk=5)
  118. # %% [markdown]
  119. # As we can see, the result is pretty reasonable.
  121. # %% [markdown]
  122. # ## 1.3 Inspect the model prediction with Lime
  124. # %% [markdown]
  125. # In this section, we will bring in LIME from Captum to analyze how the Resnet made the above prediction based on the sample image.
  126. #
  127. # Like many other Captum algorithms, Lime also supports analyzing a number of input features together as a group. This is very useful when dealing with images, where each color channel in each pixel is an input feature. Such a group is also refered as "super-pixel". To define our desired groups over input features, all we need is to provide a feature mask.
  128. #
  129. # In case of an image input, the feature mask is a 2D image of the same size, where each pixel in the mask indicates the feature group it belongs to via an integer value. Pixels of the same value define a group.
  130. #
  131. # This means we can readily use VOC's segmentation masks as feature masks for Captum! However, while segmentaion numbers range from 0 to 255, Captum prefers consecutive group IDs for efficiency. Therefore, we will also include extra steps to convert mask IDs.
  133. # %%
  134. seg_ids = sorted(seg_mask.unique().tolist())
  135. print('Segmentation IDs:', seg_ids)
  137. # map segment IDs to feature group IDs
  138. feature_mask = seg_mask.clone()
  139. for i, seg_id in enumerate(seg_ids):
  140. feature_mask[feature_mask == seg_id] = i
  142. print('Feature mask IDs:', feature_mask.unique().tolist())
  144. # %% [markdown]
  145. # It is time to configure our Lime algorithm. Essentially, Lime trains an interpretable surrogate model to simulate the target model's predictions. So, building an appropriate interpretable model is the most critical step in Lime. Fortunately, Captum has provided many of the most common interpretable models to save the efforts. We will demonstrate the usages of Linear Regression and Linear Lasso. Another important factor is the similarity function. Because Lime aims to explain the local behavior of an example, it will reweight the training samples according to their similarity distances. By default, Captum's Lime uses the exponential kernel on top of the cosine distance. We will change to Euclidean distance instead which is more popular in vision.
  147. # %%
  148. exp_eucl_distance = get_exp_kernel_similarity_function('euclidean', kernel_width=1000)
  150. lr_lime = Lime(
  151. resnet,
  152. interpretable_model=SkLearnLinearRegression(), # build-in wrapped sklearn Linear Regression
  153. similarity_func=exp_eucl_distance
  154. )
  156. # %% [markdown]
  157. # Next, we will analyze these groups' influence on the most confident prediction `television`. Every time we call Lime's `attribute` function, an interpretable model is trained around the given input, so unlike many other Captum's attribution algorithms, it is strongly recommended to only provide a single example as input (tensors with first dimension or batch size = 1). There are advanced use cases of passing batched inputs. Interested readers can check the [documentation](https://captum.ai/api/lime.html) for details.
  158. #
  159. # In order to train the interpretable model, we need to specify enough training data through the argument `n_samples`. Lime creates the perturbed samples in the form of interpretable representation, i.e., a binary vector indicating the “presence” or “absence” of features. Lime needs to keep calling the target model to get the labels/values for all perturbed samples. This process can be quite time-consuming depending on the complexity of the target model and the number of samples. Setting the `perturbations_per_eval` can batch multiple samples in one forward pass to shorten the process as long as your machine still has capacity. You may also consider turning on the flag `show_progress` to display a progess bar showing how many forward calls are left.
  161. # %%
  162. label_idx = output_probs.argmax().unsqueeze(0)
  163. attrs = lr_lime.attribute(
  164. img.unsqueeze(0),
  165. target=label_idx,
  166. feature_mask=feature_mask.unsqueeze(0),
  167. n_samples=40,
  168. perturbations_per_eval=16,
  169. show_progress=True
  170. ).squeeze(0)
  172. print('Attribution range:', attrs.min().item(), 'to', attrs.max().item())
  174. # %% [markdown]
  175. # Now, let us use Captum's visualization tool to view the attribution heat map.
  177. # %%
  178. def show_attr(attr_map):
  179. viz.visualize_image_attr(
  180. attr_map.permute(1, 2, 0).numpy(), # adjust shape to height, width, channels
  181. method='heat_map',
  182. sign='all',
  183. show_colorbar=True
  184. )
  186. show_attr(attrs)
  188. # %% [markdown]
  189. # The result looks decent: the television segment does demonstrate strongest positive correlation with the prediction, while the chairs have relatively trivial impact and the border slightly shows negative contribution.
  190. #
  191. # However, we can further improve this result. One desired characteristic of interpretability is the ease for humans to comprehend. We should help reduce the noisy interference and emphisze the real influential features. In our case, all features more or less show some influences. Adding lasso regularization to the interpretable model can effectively help us filter them. Therefore, let us try Linear Lasso with a fit coefficient `alpha`. For all built-in sklearn wrapper models, you can directly pass any sklearn supported arguments.
  192. #
  193. # Moreover, since our example only has 4 segments, there are just 16 possible combinations of interpretable representations in total. So we can exhaust them instead of random sampling. The `Lime` class's argument `perturb_func` allows us to pass a generator function yielding samples. We will create the generator function iterating the combinations and set the `n_samples` to its exact length.
  195. # %%
  196. n_interpret_features = len(seg_ids)
  198. def iter_combinations(*args, **kwargs):
  199. for i in range(2 ** n_interpret_features):
  200. yield torch.tensor([int(d) for d in bin(i)[2:].zfill(n_interpret_features)]).unsqueeze(0)
  202. lasso_lime = Lime(
  203. resnet,
  204. interpretable_model=SkLearnLasso(alpha=0.08),
  205. similarity_func=exp_eucl_distance,
  206. perturb_func=iter_combinations
  207. )
  209. attrs = lasso_lime.attribute(
  210. img.unsqueeze(0),
  211. target=label_idx,
  212. feature_mask=feature_mask.unsqueeze(0),
  213. n_samples=2 ** n_interpret_features,
  214. perturbations_per_eval=16,
  215. show_progress=True
  216. ).squeeze(0)
  218. print('Attribution range:', attrs.min().item(), 'to', attrs.max().item())
  219. show_attr(attrs)
  221. # %% [markdown]
  222. # As we can see, the new attribution result removes the chairs and border with the help of Lasso.
  223. #
  224. # Another interesting question to explore is if the model also recognizes the chairs in the image. To answer this, we will use the most related label `rocking_chair` from ImageNet as the target, whose label index is `765`. We can check how confident the model feels about the alternative object.
  226. # %%
  227. alter_label_idx = 765
  229. alter_prob = output_probs[alter_label_idx].item()
  230. print(f'{idx_to_labels[str(alter_label_idx)]} ({alter_label_idx}):', round(alter_prob, 4))
  232. # %% [markdown]
  233. # Then, we will redo the attribution with our Lasso Lime.
  235. # %%
  236. attrs = lasso_lime.attribute(
  237. img.unsqueeze(0),
  238. target=alter_label_idx,
  239. feature_mask=feature_mask.unsqueeze(0),
  240. n_samples=2 ** n_interpret_features,
  241. perturbations_per_eval=16,
  242. show_progress=True,
  243. return_input_shape=True,
  244. ).squeeze(0)
  246. print('Attribution range:', attrs.min().item(), 'to', attrs.max().item())
  247. show_attr(attrs)
  249. # %% [markdown]
  250. # As shown in the heat map, our ResNet does present the right belief about the chair segment. However, it gets hindered by the television segment in the foreground. This may also explain why the model feels less confident about the chairs than the television.
  252. # %% [markdown]
  253. # ### 1.4 Understand the sampling process
  255. # %% [markdown]
  256. # We have already learned how to use Captum's Lime. This section will additionally dive into the internal sampling process to give interested readers an overview of what happens underneath. The goal of the sampling process is to collect a set of training data for the surrogate model. Every data point consists of three parts: interpretable input, model predicted label, and similarity weight. We will roughly illustrate how Lime achieves each of them behind the scenes.
  257. #
  258. # As we mentioned before, Lime samples data from the interpretable space. By default, Lime uses the presence or absence of the given mask groups as interpretable features. In our example, facing the above image of 4 segments, the interpretable representation is therefore a binary vector of 4 values indicating if each segment is present or absent. This is why we know there are only 16 possible interpretable representations and can exhaust them with our `iter_combinations`. Lime will keep calling its `perturb_func` to get the sample interpretable inputs. Let us simulate this step and give us such an interpretable input.
  260. # %%
  261. SAMPLE_INDEX = 13
  263. pertubed_genertator = iter_combinations()
  264. for _ in range(SAMPLE_INDEX + 1):
  265. sample_interp_inp = next(pertubed_genertator)
  267. print('Perturbed interpretable sample:', sample_interp_inp)
  269. # %% [markdown]
  270. # Our input sample `[1, 1, 0, 1]` means the third segment (television) is absent while other three segments stay.
  271. #
  272. # In order to find out what the target ImageNet's prediction is for this sample, Lime needs to convert it from interpretable space back to the original example space, i.e., the image space. The transformation takes the original example input and modifies it by setting the features of the absent groups to a baseline value which is `0` by default. The transformation function is called `from_interp_rep_transform` under Lime. We will run it manually here to get the pertubed image input and then visualize what it looks like.
  274. # %%
  275. pertubed_img = lasso_lime.from_interp_rep_transform(
  276. sample_interp_inp,
  277. img.unsqueeze(0),
  278. feature_mask=feature_mask.unsqueeze(0),
  279. baselines=0
  280. )
  282. # invert the normalization for render
  283. invert_norm = T.Normalize(
  284. mean=[-0.485/0.229, -0.456/0.224, -0.406/0.225],
  285. std=[1/0.229, 1/0.224, 1/0.225]
  286. )
  288. plt.imshow(invert_norm(pertubed_img).squeeze(0).permute(1, 2, 0).numpy())
  289. plt.axis('off')
  290. plt.show()
  292. # %% [markdown]
  293. # As shown above, compared with the original image, the absent feature, i.e., the television segment, gets masked in the perturbed image, while the other present features stay unchanged. With the perturbed image, Lime is able to find out the model's prediction. Let us still use "television" as our attribution target, so the label of perturbed sample is the value of the model's prediction on "television". Just for curiosity, we can also check how the model's prediction changes with the perturbation.
  295. # %%
  296. perturbed_outputs = resnet(pertubed_img).squeeze(0).detach()
  297. sample_label = perturbed_outputs[label_idx.item()]
  298. print('Label of the perturbed sample as Television:', sample_label)
  300. print('\nProbabilities of the perturbed image')
  301. perturbed_output_probs = F.softmax(perturbed_outputs, dim=0)
  302. print_result(perturbed_output_probs, topk=5)
  303. print(f'\ntelevision ({label_idx.item()}):', perturbed_output_probs[label_idx].item())
  305. # %% [markdown]
  306. # Reasonably, our ImageNet no longer feel confident about classifying the image as a television.
  307. #
  308. # At last, because Lime focuses on the local interpretability, it will calculate the similarity between the perturbed and original images to reweight the loss of this data point. Note the calculation is based on the input space instead of the interpretable space. This step is simply passing the two image tensors into the given `similarity_func` argument which is the exponential kernel of Euclidean distance in our case.
  310. # %%
  311. sample_similarity = exp_eucl_distance(img.unsqueeze(0), pertubed_img, None)
  312. print('Sample similarity:', sample_similarity)
  314. # %% [markdown]
  315. # This is basically how Lime create a single training data point of `sample_interp_inp`, `sample_label`, and `sample_similarity`. By repeating this process `n_samples` times, it collects a dataset to train the interpretable model.
  316. #
  317. # It is Worth noting that the steps we showed in this section is an example based on our Lime instance configured above. The logic of each step can be customized, especially with `LimeBase` class which will be demonstrated in Section 2.
  319. # %% [markdown]
  320. # ## 2. Text Classification
  322. # %% [markdown]
  323. # In this section, we will take use of a news subject classification example to demonstrate more customizable functions in Lime. We will train a simple embedding-bag classifier on the AG_NEWS dataset and analyze its understanding of words.
  325. # %%
  326. from torch import nn
  327. from torch.utils.data import DataLoader
  328. from torch.utils.data.dataset import random_split
  329. from torchtext.datasets import AG_NEWS
  330. from torchtext.data.utils import get_tokenizer
  331. from torchtext.vocab import Vocab
  333. from collections import Counter
  335. from IPython.core.display import HTML, display
  337. # %% [markdown]
  338. # ### 2.1 Load the data and define the model
  340. # %% [markdown]
  341. # `torchtext` has included the AG_NEWS dataset but since it is only split into train & test, we need to further cut a validation set from the original train split. Then we build the vocabulary of the frequent words based on our train split.
  343. # %%
  344. ag_ds = list(AG_NEWS(split='train'))
  346. ag_train, ag_val = ag_ds[:100000], ag_ds[100000:]
  348. tokenizer = get_tokenizer('basic_english')
  349. word_counter = Counter()
  350. for (label, line) in ag_train:
  351. word_counter.update(tokenizer(line))
  352. voc = Vocab(word_counter, min_freq=10)
  354. print('Vocabulary size:', len(voc))
  356. num_class = len(set(label for label, _ in ag_train))
  357. print('Num of classes:', num_class)
  359. # %% [markdown]
  360. # The model we use is composed of an embedding-bag, which averages the word embeddings as the latent text representation, and a final linear layer, which maps the latent vector to the logits. Unconventially, `pytorch`'s embedding-bag does not assume the first dimension is batch. Instead, it requires a flattened vector of indices with an additional offset tensor to mark the starting position of each example. You can refer to its [documentation](https://pytorch.org/docs/stable/generated/torch.nn.EmbeddingBag.html#embeddingbag) for details.
  362. # %%
  363. class EmbeddingBagModel(nn.Module):
  364. def __init__(self, vocab_size, embed_dim, num_class):
  365. super().__init__()
  366. self.embedding = nn.EmbeddingBag(vocab_size, embed_dim)
  367. self.linear = nn.Linear(embed_dim, num_class)
  369. def forward(self, inputs, offsets):
  370. embedded = self.embedding(inputs, offsets)
  371. return self.linear(embedded)
  373. # %% [markdown]
  374. # ### 2.2 Training and Baseline Classification
  376. # %% [markdown]
  377. # In order to train our classifier, we need to define a collate function to batch the samples into the tensor fomat required by the embedding-bag and create the interable dataloaders.
  379. # %%
  380. BATCH_SIZE = 64
  382. def collate_batch(batch):
  383. labels = torch.tensor([label - 1 for label, _ in batch])
  384. text_list = [tokenizer(line) for _, line in batch]
  386. # flatten tokens across the whole batch
  387. text = torch.tensor([voc[t] for tokens in text_list for t in tokens])
  388. # the offset of each example
  389. offsets = torch.tensor(
  390. [0] + [len(tokens) for tokens in text_list][:-1]
  391. ).cumsum(dim=0)
  393. return labels, text, offsets
  395. train_loader = DataLoader(ag_train, batch_size=BATCH_SIZE,
  396. shuffle=True, collate_fn=collate_batch)
  397. val_loader = DataLoader(ag_val, batch_size=BATCH_SIZE,
  398. shuffle=False, collate_fn=collate_batch)
  400. # %% [markdown]
  401. # We will then train our embedding-bag model with the common cross-entropy loss and Adam optimizer. Due to the simplicity of this task, 5 epochs should be enough to give us a stable 90% validation accuracy.
  403. # %%
  404. EPOCHS = 7
  405. EMB_SIZE = 64
  406. CHECKPOINT = './models/embedding_bag_ag_news.pt'
  407. USE_PRETRAINED = True # change to False if you want to retrain your own model
  409. def train_model(train_loader, val_loader):
  410. model = EmbeddingBagModel(len(voc), EMB_SIZE, num_class)
  412. loss = nn.CrossEntropyLoss()
  413. optimizer = torch.optim.Adam(model.parameters())
  415. for epoch in range(1, EPOCHS + 1):
  416. # training
  417. model.train()
  418. total_acc, total_count = 0, 0
  420. for idx, (label, text, offsets) in enumerate(train_loader):
  421. optimizer.zero_grad()
  422. predited_label = model(text, offsets)
  423. loss(predited_label, label).backward()
  424. optimizer.step()
  425. total_acc += (predited_label.argmax(1) == label).sum().item()
  426. total_count += label.size(0)
  428. if (idx + 1) % 500 == 0:
  429. print('epoch {:3d} | {:5d}/{:5d} batches | accuracy {:8.3f}'.format(
  430. epoch, idx + 1, len(train_loader), total_acc / total_count
  431. ))
  432. total_acc, total_count = 0, 0
  434. # evaluation
  435. model.eval()
  436. total_acc, total_count = 0, 0
  438. with torch.no_grad():
  439. for label, text, offsets in val_loader:
  440. predited_label = model(text, offsets)
  441. total_acc += (predited_label.argmax(1) == label).sum().item()
  442. total_count += label.size(0)
  444. print('-' * 59)
  445. print('end of epoch {:3d} | valid accuracy {:8.3f} '.format(epoch, total_acc / total_count))
  446. print('-' * 59)
  448. torch.save(model, CHECKPOINT)
  449. return model
  451. eb_model = torch.load(CHECKPOINT, weights_only=False) if USE_PRETRAINED else train_model(train_loader, val_loader)
  453. # %% [markdown]
  454. # Now, let us take the following sports news and test how our model performs.
  456. # %%
  457. test_label = 2 # {1: World, 2: Sports, 3: Business, 4: Sci/Tec}
  458. test_line = ('US Men Have Right Touch in Relay Duel Against Australia THENS, Aug. 17 '
  459. '- So Michael Phelps is not going to match the seven gold medals won by Mark Spitz. '
  460. 'And it is too early to tell if he will match Aleksandr Dityatin, '
  461. 'the Soviet gymnast who won eight total medals in 1980.')
  463. test_labels, test_text, test_offsets = collate_batch([(test_label, test_line)])
  465. probs = F.softmax(eb_model(test_text, test_offsets), dim=1).squeeze(0)
  466. print('Prediction probability:', round(probs[test_labels[0]].item(), 4))
  468. # %% [markdown]
  469. # Our embedding-bag does successfully identify the above news as sports with pretty high confidence.
  471. # %% [markdown]
  472. # ### 2.3 Inspect the model prediction with Lime
  474. # %% [markdown]
  475. # Finally, it is time to bring back Lime to inspect how the model makes the prediction. However, we will use the more customizable `LimeBase` class this time which is also the low-level implementation powering the `Lime` class we used before. The `Lime` class is opinionated when creating features from perturbed binary interpretable representations. It can only set the "absence" features to some baseline values while keeping other "presence" features. This is not what we want in this case. For text, the interpretable representation is a binary vector indicating if the word of each position is present or not. The corresponding text input should literally remove the absent words so our embedding-bag can calculate the average embeddings of the left words. Setting them to any baselines will pollute the calculation and moreover, our embedding-bag does not have common baseline tokens like `<padding>` at all. Therefore, we have to use `LimeBase` to customize the conversion logic through the `from_interp_rep_transform` argument.
  476. #
  477. # `LimeBase` is not opinionated at all so we have to define every piece manually. Let us talk about them in order:
  478. # - `forward_func`, the forward function of the model. Notice we cannot pass our model directly since Captum always assumes the first dimension is batch while our embedding-bag requires flattened indices. So we will add the dummy dimension later when calling `attribute` and make a wrapper here to remove the dummy dimension before giving to our model.
  479. # - `interpretable_model`, the surrogate model. This works the same as we demonstrated in the above image classification example. We also use sklearn linear lasso here.
  480. # - `similarity_func`, the function calculating the weights for training samples. The most common distance used for texts is the cosine similarity in their latent embedding space. The text inputs are just sequences of token indices, so we have to leverage the trained embedding layer from the model to encode them to their latent vectors. Due to this extra encoding step, we cannot use the util `get_exp_kernel_similarity_function('cosine')` like in the image classification example, which directly calculate the cosine similarity of the given inputs.
  481. # - `perturb_func`, the function to sample interpretable representations. We present another way to define this argument other than using generator as shown in the above image classification example. Here we directly define a function returning a randomized sample every call. It outputs a binary vector where each token is selected independently and uniformly at random.
  482. # - `perturb_interpretable_space`, whether perturbed samples are in interpretable space. `LimeBase` also supports sampling in the original input space, but we do not need it in our case.
  483. # - `from_interp_rep_transform`, the function transforming the perturbed interpretable samples back to the original input space. As explained above, this argument is the main reason for us to use `LimeBase`. We pick the subset of the present tokens from the original text input according to the interpretable representation.
  484. # - `to_interp_rep_transform`, the opposite of `from_interp_rep_transform`. It is needed only when `perturb_interpretable_space` is set to false.
  486. # %%
  487. # remove the batch dimension for the embedding-bag model
  488. def forward_func(text, offsets):
  489. return eb_model(text.squeeze(0), offsets)
  491. # encode text indices into latent representations & calculate cosine similarity
  492. def exp_embedding_cosine_distance(original_inp, perturbed_inp, _, **kwargs):
  493. original_emb = eb_model.embedding(original_inp, None)
  494. perturbed_emb = eb_model.embedding(perturbed_inp, None)
  495. distance = 1 - F.cosine_similarity(original_emb, perturbed_emb, dim=1)
  496. return torch.exp(-1 * (distance ** 2) / 2)
  498. # binary vector where each word is selected independently and uniformly at random
  499. def bernoulli_perturb(text, **kwargs):
  500. probs = torch.ones_like(text) * 0.5
  501. return torch.bernoulli(probs).long()
  503. # remove absent token based on the intepretable representation sample
  504. def interp_to_input(interp_sample, original_input, **kwargs):
  505. return original_input[interp_sample.bool()].view(original_input.size(0), -1)
  507. lasso_lime_base = LimeBase(
  508. forward_func,
  509. interpretable_model=SkLearnLasso(alpha=0.08),
  510. similarity_func=exp_embedding_cosine_distance,
  511. perturb_func=bernoulli_perturb,
  512. perturb_interpretable_space=True,
  513. from_interp_rep_transform=interp_to_input,
  514. to_interp_rep_transform=None
  515. )
  517. # %% [markdown]
  518. # The attribution call is the same as the `Lime` class. Just remember to add the dummy batch dimension to the text input and put the offsets in the `additional_forward_args` because it is not a feature for the classification but a metadata for the text input.
  520. # %%
  521. attrs = lasso_lime_base.attribute(
  522. test_text.unsqueeze(0), # add batch dimension for Captum
  523. target=test_labels,
  524. additional_forward_args=(test_offsets,),
  525. n_samples=32000,
  526. show_progress=True
  527. ).squeeze(0)
  529. print('Attribution range:', attrs.min().item(), 'to', attrs.max().item())
  531. # %% [markdown]
  532. # At last, let us create a simple visualization to highlight the influential words where green stands for positive correlation and red for negative.
  534. # %%
  535. def show_text_attr(attrs):
  536. rgb = lambda x: '255,0,0' if x < 0 else '0,255,0'
  537. alpha = lambda x: abs(x) ** 0.5
  538. token_marks = [
  539. f'<mark style="background-color:rgba({rgb(attr)},{alpha(attr)})">{token}</mark>'
  540. for token, attr in zip(tokenizer(test_line), attrs.tolist())
  541. ]
  543. display(HTML('<p>' + ' '.join(token_marks) + '</p>'))
  545. show_text_attr(attrs)
  547. # %% [markdown]
  548. # The above visulization should render something like the image below where the model links the "Sports" subject to many reasonable words, like "match" and "medals".
  549. #
  550. # ![Lime Text](img/lime_text_viz.png)

Image_and_Text_Classification_LIME.ipynb at commit 2a3c3df, under BSD-3-Clause · at the source

Overview

Authors: Erik Kaestner1, Jay Sawant2, Donatello Arienzo1, Kyle A Hasenstab3, Ezequiel Gleichgerrcht4, Taha Gholipour5, Anees Abrol6, Reihaneh Hassanzadeh7,8, Sophia I Thomopoulos9, Clarissa L Yasuda10,11, Lucas Scárdua Silva12, Marina K M Alvim10,11, Patrick Moloney13,14,15, Andre Altmann16, Helena Martins Custodio13, Ev-Christin Heide17,18, Nishant Sinha19, Alice Ballerini20, Julie Absil21, Sara Larivière22
and 49 other authorsKai M Schubert23, Carolina Ferreira-Atuesta24, Gian Marco Duma25, Raphaël Christin26, Elisa Barbi27, Renzo Guerrini27,28, Theodor Rüber29,30,31,32, Tobias Bauer29,30,31, Benjamin Sinclair33, Jacob Bunyamin33, Merran R Courtney33,34, Meng Law33, Angelo Labate35, Pasquale Striano36,37, Lucy Vivash33,34, Terence J O’Brien33,38, Matteo Lenge27, Luca Saba39, Jonathan K Kleen26, Paolo Bonanni25, Leigh N Sepeta40, Marian Galovic24, Emanuele Bartolini41, Victoria Ives-Deliperi42, Boris C Bernhardt43, Pascal Martin44, Chantal Depondt45, Travis Stoub46, Anna Elisabetta Vaudano47, Stefano Meletti47,48, Ruben Kuzniecky49, Luis Concha50, Anto I Bagić51, Kathryn A Davis19,52, Richard J Staba53, Niels K N Focke54, Heath Pardoe55,56, Patricia C Dugan55, Orrin Devinsky57,58, Daniel L Drane4, Zhiqiang Zhang59, Antonio Gambardella60,61, Alexandra Parashos62, Fernando Cendes63,64, Paul M Thompson9, Sanjay M Sisodiya13,65, Vince D Calhoun66, Leonardo Bonilha67, Carrie R McDonald1,68
ORCID iDs: Erik Kaestner, Andre Altmann, Helena Martins Custodio, Nishant Sinha, Alice Ballerini, Kai M Schubert, Gian Marco Duma, Theodor Rüber, Tobias Bauer, Lucy Vivash, Matteo Lenge, Jonathan K Kleen, Marian Galovic, Anna Elisabetta Vaudano, Stefano Meletti, Luis Concha, Kathryn A Davis, Richard J Staba, Orrin Devinsky, Antonio Gambardella, Sanjay M Sisodiya
68 affiliations
  1. Department of Radiation Medicine and Applied Sciences, University of California, San Diego, La Jolla, CA 92037, USA
  2. Halıcıoğlu Data Science Institute, University of California, San Diego, La Jolla, CA 92093, USA
  3. Department of Mathematics and Statistics, San Diego State University, San Diego, CA 92182, USA
  4. Department of Neurology, Emory University, Atlanta, GA 30322, USA
  5. Department of Neurosciences, University of California, San Diego, La Jolla, CA 92093, USA
  6. GSU/GATech/Emory Center for Translational Research in Neuroimaging and Data Science (TReNDS), Atlanta, GA 30303, USA
  7. Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
  8. Center for Translational Research in Neuroimaging and Data Science, Georgia State University, Atlanta, GA 30303, USA
  9. Imaging Genetics Center, Mark and Mary Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Marina del Rey, CA 90292, USA
  10. Brazilian Institute of Neuroscience and Neurotechnology (BRAINN), Campinas, SP 13083-970, Brazil
  11. Department of Neurology, University of Campinas (UNICAMP), Campinas, SP 13083-888, Brazil
  12. Neuroimaging Laboratory, University of Campinas (UNICAMP), Campinas, SP 13083-888, Brazil
  13. Department of Clinical and Experimental Epilepsy, UCL Queen Square Institute of Neurology, London WC1N 3BG, UK
  14. Dublin Neurological Institute, Mater Misericordiae University Hospital, Dublin D07 W7XF, Ireland
  15. School of Medicine, University College Dublin, Dublin D04 V1W8, Ireland
  16. Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK
  17. Department of Neurology, University Medical Center Göttingen, Göttingen 37075, Germany
  18. Department of Psychiatry and Psychotherapy, University of Cologne, Cologne 50937, Germany
  19. Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA
  20. Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Baggiovara 41126, Italy
  21. Department of Radiology, CUB Erasme Hospital, Hôpital Universitaire de Bruxelles, Université Libre de Bruxelles, Brussels 1070, Belgium
  22. Department of Medical Imaging and Radiation Sciences, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, QC J1H 5N4, Canada
  23. Department of Neurology, Clinical Neuroscience Center, University Hospital and University of Zurich, Zurich 8091, Switzerland
  24. Department of Neurology, University Hospital Zurich, Zurich 8091, Switzerland
  25. Epilepsy and Clinical Neurophysiology Unit, Scientific Institute IRCCS E. Medea, Conegliano 31015, Italy
  26. Department of Neurology, University of California SanFrancisco, San Francisco, CA 94158, USA
  27. Department of Neuroscience and Human Genetics, Meyer Children’s Hospital IRCCS, Florence 50139, Italy
  28. Department of Neuroscience, Pharmacology and Child Health, University of Florence, Florence 50139, Italy
  29. Department of Neuroradiology, University Hospital Bonn, Bonn 53127, Germany
  30. Department of Epileptology, University Hospital Bonn, Bonn 53127, Germany
  31. German Center for Neurodegenerative Diseases (DZNE), Bonn 53127, Germany
  32. Center for Medical Data Usability and Translation, University of Bonn, Bonn 53127, Germany
  33. Department of Neuroscience, School of Translational Medicine, Alfred Health, Monash University, Melbourne, VIC 53127, Australia
  34. Department of Neurology, Alfred Health, Melbourne, VIC 3004, Australia
  35. Neurophysiopathology and Movement Disorders Clinic, University of Messina, Messina 98125, Italy
  36. IRCCS G. Gaslini, Full Member of Epicare, Genova 16147, Italy
  37. Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health, University of Genova, Genova 16132, Italy
  38. Departments of Medicine and Neurology, the Royal Melbourne Hospital, The University of Melbourne, Parkville, VIC 3050, Australia
  39. Department of Radiology, AOU Cagliari and University of Cagliari, Cagliari 09042, Italy
  40. Children’s National Hospital (CNH), Washington, DC 20010, USA
  41. Department of Developmental Neuroscience, IRCCS Foundation Stella Maris, Pisa 56128, Italy
  42. Department of Psychiatry, Neuroscience Institute, University of Cape Town, Cape Town 7925, South Africa
  43. Centre of Excellence in Epilepsy at the Neuro and McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill University, Montreal, QC H3A 2B4, Canada
  44. Department of Neurology and Epileptology, Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen 72076, Germany
  45. Department of Neurology, CUB Erasme Hospital, Hôpital Universitaire de Bruxelles, Université Libre de Bruxelles, Brussels 1070, Belgium
  46. Department of Neurological Sciences, Rush University Medical Center, Chicago, IL 60612, USA
  47. Department of Biomedical, Metabolic and Neuronal Science, University of Modena and Reggio Emilia, Modena 41125, Italy
  48. Neurophysiology Unit and Epilepsy Centre, AOU Modena, Modena 41126, Italy
  49. Department of Neurology, School of Medicine at Hofstra/Northwell, Hempstead, NY 11549, USA
  50. Institute of Neurobiology, Universidad Nacional Autónoma de México, Querétaro 76230, Mexico
  51. University of Pittsburgh Comprehensive Epilepsy Center (UPCEC), Department of Neurology, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA
  52. Center for Neuroengineering and Therapeutics, University of Pennsylvania, Philadelphia, PA 19104, United States
  53. Department of Neurology, David Geffen School of Medicine at UCLA, Los Angeles, CA 90095, USA
  54. Clinic for Neurology, University Medical Center Göttingen, Göttingen 37075, Germany
  55. Department of Neurology, NYU Grossman School of Medicine, NewYork, NY 10016, USA
  56. Florey Institute of Neuroscience and Mental Health, Heidelberg, VIC 3084, Australia
  57. NYU Comprehensive Epilepsy Center, NewYork, NY 10016, USA
  58. Department of Neurology, Langone School of Medicine, NewYork University, New York, NY 10016, USA
  59. Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing 210002, China
  60. Institute of Neurology, Magna Græcia University, Catanzaro 88100, Italy
  61. Neuroscience Research Center, Magna Græcia University, Catanzaro 88100, Italy
  62. Department of Neurology, Medical University of South Carolina, Charleston, SC 29425, USA
  63. Department of Neurology, FCM, University of Campinas—UNICAMP, Campinas, SP 13083-888, Brazil
  64. Brazilian Institute of Neuroscience and Neurotechnology, Campinas, SP 13083-970, Brazil
  65. Chalfont Centre for Epilepsy, Chalfont St Peter, Bucks SL9 0RJ, UK
  66. Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory, Atlanta, GA 30303, USA
  67. Department of Neurology, University of South Carolina, Columbia, SC 29203, USA
  68. Department of Psychiatry, University of California, San Diego, La Jolla, CA 92037, USA
Institutions: University of California San Diego (United States); San Diego State University (United States); Emory University (United States); Georgia State University (United States); Center for Translational Research in Neuroimaging and Data Science (United States); Georgia Institute of Technology (United States); University of Southern California (United States); Universidade Estadual de Campinas (UNICAMP) (Brazil); Brazilian Institute of Neuroscience and Neurotechnology (Brazil); University College Dublin (Ireland); Mater Misericordiae University Hospital (Ireland); UCL Queen Square Institute of Neurology (United Kingdom); University College London (United Kingdom); University of Cologne (Germany); Universitätsmedizin Göttingen (Germany); University of Göttingen (Germany); University of Pennsylvania (United States); University of Modena and Reggio Emilia (Italy); Université Libre de Bruxelles (Belgium); Erasmus Hospital (Belgium); Université de Sherbrooke (Canada); University of Zurich (Switzerland); University Hospital Zurich (Switzerland); IRCCS Eugenio Medea (Italy); University of California, San Francisco (United States); Meyer Children's Hospital (Italy); University of Florence (Italy); University of Bonn (Germany); University Hospital Bonn (Germany); German Center for Neurodegenerative Diseases (Germany); Alfred Health (Australia); Monash University (Australia); The Alfred Hospital (Australia); University of Messina (Italy); Istituto Giannina Gaslini (Italy); University of Genoa (Italy); The Royal Melbourne Hospital (Australia); The University of Melbourne (Australia); University of Cagliari (Italy); Children's National (United States); Fondazione Stella Maris (Italy); University of Cape Town (South Africa); Montreal Neurological Institute and Hospital (Canada); McGill University (Canada); Hertie Institute for Clinical Brain Research (Germany); University of Tübingen (Germany); Rush University Medical Center (United States); Hofstra University (United States); Donald & Barbara Zucker School of Medicine at Hofstra/Northwell (United States); Universidad Nacional Autónoma de México (Mexico); University of Pittsburgh (United States); University of California, Los Angeles (United States); Florey Institute of Neuroscience and Mental Health (Australia); New York University (United States); NYU Langone Health (United States); Nanjing General Hospital of Nanjing Military Command (China); Nanjing University (China); Magna Graecia University (Italy); Medical University of South Carolina (United States); University of South Carolina (United States)
Journal: Brain communications, volume 8, issue 4, article fcag253
Dates: received 6 November 2025; accepted 27 May 2026; published online 29 June 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1093/braincomms/fcag253 · PMID 42534493 · PMCID PMC13421366 · OpenAlex W7166573953
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: structural MRI / diffusion (modality), human (organism), epilepsy (population), clinical / translational (subfield)
Methods: Connectivity, Statistics, Machine learning, Preprocessing, fMRI & imaging
Keywords: epilepsy, diagnosis, lateralization, AI, MRI
Topic: Epilepsy research and treatment (Psychiatry and Mental health, Medicine), according to OpenAlex
Funding: National Institutes of Health Big Data to Knowledge (U54 EB020403)
Citations: cited by 1 paper (Europe PMC); 41 references in the paper

Abstract

Diagnostic MRI evaluation of temporal lobe epilepsy (TLE) depends on the subjective visual interpretation of MRI images. These interpretations could be enhanced by quantitative artificial intelligence (AI) support tools. Humans often make sequential and conditional decisions during their radiological interpretations, such as whether an abnormality is present and, if present, characterizing the abnormality. It is not known whether it is superior to train AI to treat every decision separately in a similar step-wise manner or to train a model holistically on all decisions simultaneously. Here, we analysed three large epilepsy MRI datasets [n = 3676, 2320 people with epilepsy and 1356 healthy controls (HC)] to perform two tasks: (i) establish the presence of a TLE pattern on MRI and (ii) determine TLE pattern lateralization. We compared Step-wise models that independently classify TLE versus HC and lateralize patients as left TLE (L-TLE) or right TLE (R-TLE), against a simultaneous model trained to distinguish all three classes in a single step. To do this, 3D volumetric T1-weighted images were input into an EfficientNetV2 model multiple times to ensure reproducibility of results. Class prediction, model classification confidence and saliency maps were output for interpretability. Step-wise models outperformed the Simultaneous model on both tasks (both Ps < 0.001), with an average ∼2.8% accuracy increase for discriminating HC from TLE and an average 12.7% accuracy increase for distinguishing L-TLE from R-TLE. For both the Step-wise and Simultaneous models, important features discriminating TLE from HC included the known TLE limbic pattern involving the hippocampus, parahippocampal cortical regions, cingulate cortex and lateral temporal regions. However, there was less concordance between the Step-wise and Simultaneous models for the L-TLE versus R-TLE task (all Fisher’s Zs > 10.5, Ps < 0.001); the Step-wise model focused less on subcortical regions such as the thalamus and hippocampus and focused more on distributed cortical pathology. Across the two Step-wise models, 95.1% of TLE patients had accurate classifications in either HC versus TLE and/or L-TLE versus R-TLE tasks. These results included 69.6% of patients being both correctly labelled as TLE and lateralized, 13.9% being correctly labelled TLE but lateralized incorrectly and 11.6% being lateralized correctly but not detected as TLE. These findings provide evidence that diagnostic tasks with simpler, Step-wise AI models may enhance diagnostic performance and interpretability in clinical workflows. Future AI clinical support tools can leverage this step-wise approach in the early identification of TLE-related structural patterns, supporting timely diagnosis and treatment decisions.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.

meta-pytorch/captum

License: BSD-3-Clause
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 2a3c3dfe660bd23def7de91db87bc2aab5708341, 27 September 2026
Languages: Python (241), Jupyter (20), JavaScript (8), Shell (3)
Size: 393 files, 272 scripts
Software Heritage: archived
Found in: the text, “Feature visualization”
Holds: README, license file, environment (pyproject.toml), tests, continuous integration, documentation, 20 notebooks
Not found: CITATION.cff
Tools: PyTorch (199 files), NumPy (36 files), Matplotlib (21 files), Pillow (10 files), scikit-learn (8 files), Hugging Face Transformers (6 files), pandas (5 files), SciPy (4 files), seaborn (2 files), OpenCV (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
274 files
At the source: github.com/meta-pytorch/captum

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 272 scripts, each with its path and the digest of its content;
  • 4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability

Data are available from the corresponding author from the ENIGMA-Epilepsy dataset upon reasonable request and with appropriate data-sharing agreements. Data from ECP and CAPES are available from their respective cohort PIs upon reasonable request. The code used for processing the images, modelling and visualization is noted in their respective sections.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 69 authors, 5 keywords, 1 funder, 40 references.

Cite

This paper

Kaestner, E., Sawant, J., Arienzo, D., Hasenstab, K. A., Gleichgerrcht, E., Gholipour, T., Abrol, A., Hassanzadeh, R., Thomopoulos, S. I., Yasuda, C. L., Silva, L. S., Alvim, M. K. M., Moloney, P., Altmann, A., Martins Custodio, H., Heide, E.-C., Sinha, N., Ballerini, A., Absil, J., . . . McDonald, C. R. (2026). Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study. Brain communications, 8(4), fcag253. https://doi.org/10.1093/braincomms/fcag253

BibTeX

@article{kaestner2026disease,
author = {Kaestner, Erik and Sawant, Jay and Arienzo, Donatello and Hasenstab, Kyle A and Gleichgerrcht, Ezequiel and Gholipour, Taha and Abrol, Anees and Hassanzadeh, Reihaneh and Thomopoulos, Sophia I and Yasuda, Clarissa L and Silva, Lucas Scárdua and Alvim, Marina K M and Moloney, Patrick and Altmann, Andre and Martins Custodio, Helena and Heide, Ev-Christin and Sinha, Nishant and Ballerini, Alice and Absil, Julie and Larivière, Sara and Schubert, Kai M and Ferreira-Atuesta, Carolina and Duma, Gian Marco and Christin, Raphaël and Barbi, Elisa and Guerrini, Renzo and Rüber, Theodor and Bauer, Tobias and Sinclair, Benjamin and Bunyamin, Jacob and Courtney, Merran R and Law, Meng and Labate, Angelo and Striano, Pasquale and Vivash, Lucy and O’Brien, Terence J and Lenge, Matteo and Saba, Luca and Kleen, Jonathan K and Bonanni, Paolo and Sepeta, Leigh N and Galovic, Marian and Bartolini, Emanuele and Ives-Deliperi, Victoria and Bernhardt, Boris C and Martin, Pascal and Depondt, Chantal and Stoub, Travis and Vaudano, Anna Elisabetta and Meletti, Stefano and Kuzniecky, Ruben and Concha, Luis and Bagić, Anto I and Davis, Kathryn A and Staba, Richard J and Focke, Niels K N and Pardoe, Heath and Dugan, Patricia C and Devinsky, Orrin and Drane, Daniel L and Zhang, Zhiqiang and Gambardella, Antonio and Parashos, Alexandra and Cendes, Fernando and Thompson, Paul M and Sisodiya, Sanjay M and Calhoun, Vince D and Bonilha, Leonardo and McDonald, Carrie R},
title = {{Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study}},
journal = {Brain communications},
year = {2026},
month = jun,
volume = {8},
number = {4},
pages = {fcag253},
publisher = {Oxford University Press},
issn = {2632-1297},
doi = {10.1093/braincomms/fcag253},
url = {https://doi.org/10.1093/braincomms/fcag253},
pmid = {42534493},
pmcid = {PMC13421366}
}

RIS

TY - JOUR
AU - Kaestner, Erik
AU - Sawant, Jay
AU - Arienzo, Donatello
AU - Hasenstab, Kyle A
AU - Gleichgerrcht, Ezequiel
AU - Gholipour, Taha
AU - Abrol, Anees
AU - Hassanzadeh, Reihaneh
AU - Thomopoulos, Sophia I
AU - Yasuda, Clarissa L
AU - Silva, Lucas Scárdua
AU - Alvim, Marina K M
AU - Moloney, Patrick
AU - Altmann, Andre
AU - Martins Custodio, Helena
AU - Heide, Ev-Christin
AU - Sinha, Nishant
AU - Ballerini, Alice
AU - Absil, Julie
AU - Larivière, Sara
AU - Schubert, Kai M
AU - Ferreira-Atuesta, Carolina
AU - Duma, Gian Marco
AU - Christin, Raphaël
AU - Barbi, Elisa
AU - Guerrini, Renzo
AU - Rüber, Theodor
AU - Bauer, Tobias
AU - Sinclair, Benjamin
AU - Bunyamin, Jacob
AU - Courtney, Merran R
AU - Law, Meng
AU - Labate, Angelo
AU - Striano, Pasquale
AU - Vivash, Lucy
AU - O’Brien, Terence J
AU - Lenge, Matteo
AU - Saba, Luca
AU - Kleen, Jonathan K
AU - Bonanni, Paolo
AU - Sepeta, Leigh N
AU - Galovic, Marian
AU - Bartolini, Emanuele
AU - Ives-Deliperi, Victoria
AU - Bernhardt, Boris C
AU - Martin, Pascal
AU - Depondt, Chantal
AU - Stoub, Travis
AU - Vaudano, Anna Elisabetta
AU - Meletti, Stefano
AU - Kuzniecky, Ruben
AU - Concha, Luis
AU - Bagić, Anto I
AU - Davis, Kathryn A
AU - Staba, Richard J
AU - Focke, Niels K N
AU - Pardoe, Heath
AU - Dugan, Patricia C
AU - Devinsky, Orrin
AU - Drane, Daniel L
AU - Zhang, Zhiqiang
AU - Gambardella, Antonio
AU - Parashos, Alexandra
AU - Cendes, Fernando
AU - Thompson, Paul M
AU - Sisodiya, Sanjay M
AU - Calhoun, Vince D
AU - Bonilha, Leonardo
AU - McDonald, Carrie R
TI - Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study
T2 - Brain communications
J2 - Brain Commun
PY - 2026
DA - 2026/06/29
VL - 8
IS - 4
SP - fcag253
SN - 2632-1297
PB - Oxford University Press
DO - 10.1093/braincomms/fcag253
UR - https://doi.org/10.1093/braincomms/fcag253
LA - en
ER -

CSL-JSON

{
"id": "10.1093/braincomms/fcag253",
"type": "article-journal",
"title": "Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study",
"container-title": "Brain communications",
"author": [
{
"family": "Kaestner",
"given": "Erik"
},
{
"family": "Sawant",
"given": "Jay"
},
{
"family": "Arienzo",
"given": "Donatello"
},
{
"family": "Hasenstab",
"given": "Kyle A"
},
{
"family": "Gleichgerrcht",
"given": "Ezequiel"
},
{
"family": "Gholipour",
"given": "Taha"
},
{
"family": "Abrol",
"given": "Anees"
},
{
"family": "Hassanzadeh",
"given": "Reihaneh"
},
{
"family": "Thomopoulos",
"given": "Sophia I"
},
{
"family": "Yasuda",
"given": "Clarissa L"
},
{
"family": "Silva",
"given": "Lucas Scárdua"
},
{
"family": "Alvim",
"given": "Marina K M"
},
{
"family": "Moloney",
"given": "Patrick"
},
{
"family": "Altmann",
"given": "Andre"
},
{
"family": "Martins Custodio",
"given": "Helena"
},
{
"family": "Heide",
"given": "Ev-Christin"
},
{
"family": "Sinha",
"given": "Nishant"
},
{
"family": "Ballerini",
"given": "Alice"
},
{
"family": "Absil",
"given": "Julie"
},
{
"family": "Larivière",
"given": "Sara"
},
{
"family": "Schubert",
"given": "Kai M"
},
{
"family": "Ferreira-Atuesta",
"given": "Carolina"
},
{
"family": "Duma",
"given": "Gian Marco"
},
{
"family": "Christin",
"given": "Raphaël"
},
{
"family": "Barbi",
"given": "Elisa"
},
{
"family": "Guerrini",
"given": "Renzo"
},
{
"family": "Rüber",
"given": "Theodor"
},
{
"family": "Bauer",
"given": "Tobias"
},
{
"family": "Sinclair",
"given": "Benjamin"
},
{
"family": "Bunyamin",
"given": "Jacob"
},
{
"family": "Courtney",
"given": "Merran R"
},
{
"family": "Law",
"given": "Meng"
},
{
"family": "Labate",
"given": "Angelo"
},
{
"family": "Striano",
"given": "Pasquale"
},
{
"family": "Vivash",
"given": "Lucy"
},
{
"family": "O’Brien",
"given": "Terence J"
},
{
"family": "Lenge",
"given": "Matteo"
},
{
"family": "Saba",
"given": "Luca"
},
{
"family": "Kleen",
"given": "Jonathan K"
},
{
"family": "Bonanni",
"given": "Paolo"
},
{
"family": "Sepeta",
"given": "Leigh N"
},
{
"family": "Galovic",
"given": "Marian"
},
{
"family": "Bartolini",
"given": "Emanuele"
},
{
"family": "Ives-Deliperi",
"given": "Victoria"
},
{
"family": "Bernhardt",
"given": "Boris C"
},
{
"family": "Martin",
"given": "Pascal"
},
{
"family": "Depondt",
"given": "Chantal"
},
{
"family": "Stoub",
"given": "Travis"
},
{
"family": "Vaudano",
"given": "Anna Elisabetta"
},
{
"family": "Meletti",
"given": "Stefano"
},
{
"family": "Kuzniecky",
"given": "Ruben"
},
{
"family": "Concha",
"given": "Luis"
},
{
"family": "Bagić",
"given": "Anto I"
},
{
"family": "Davis",
"given": "Kathryn A"
},
{
"family": "Staba",
"given": "Richard J"
},
{
"family": "Focke",
"given": "Niels K N"
},
{
"family": "Pardoe",
"given": "Heath"
},
{
"family": "Dugan",
"given": "Patricia C"
},
{
"family": "Devinsky",
"given": "Orrin"
},
{
"family": "Drane",
"given": "Daniel L"
},
{
"family": "Zhang",
"given": "Zhiqiang"
},
{
"family": "Gambardella",
"given": "Antonio"
},
{
"family": "Parashos",
"given": "Alexandra"
},
{
"family": "Cendes",
"given": "Fernando"
},
{
"family": "Thompson",
"given": "Paul M"
},
{
"family": "Sisodiya",
"given": "Sanjay M"
},
{
"family": "Calhoun",
"given": "Vince D"
},
{
"family": "Bonilha",
"given": "Leonardo"
},
{
"family": "McDonald",
"given": "Carrie R"
}
],
"container-title-short": "Brain Commun",
"volume": "8",
"issue": "4",
"page": "fcag253",
"DOI": "10.1093/braincomms/fcag253",
"PMID": "42534493",
"PMCID": "PMC13421366",
"ISSN": "2632-1297",
"publisher": "Oxford University Press",
"URL": "https://doi.org/10.1093/braincomms/fcag253",
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
29
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1007/s00234-026-04103-8 [code]
Enhanced detection of subtle cortical abnormalities in focal epilepsy using 7 T MRI surface-based models and graph neural networks.
Journal: Neuroradiology
In common: Pillow, PyTorch, seaborn, 5 other tools, epilepsy, structural MRI / diffusion, 2 references, author Matteo Lenge
[2] doi:10.1002/ana.78251
Continuous Versus Short EEG After Ischemic Stroke: What cEEG Adds for Detecting Abnormalities and Predicting Post-Stroke Epilepsy.
Journal: Annals of neurology
In common: epilepsy, clinical / translational, 2 authors
[3] doi:10.1038/s43856-026-01817-x [code]
Visual prompt engineering for multimodal and irregularly sampled medical data.
Journal: Communications medicine
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools, clinical / translational
[4] doi:10.1038/s41598-026-50791-w [code]
Hybrid Vi+ECNN framework for advanced ADHD diagnostic accuracy in medical imaging.
Journal: Scientific reports
In common: Hugging Face Transformers, OpenCV, Pillow, 6 other tools, structural MRI / diffusion, clinical / translational
[5] doi:10.1038/s41593-026-02258-4 [code]
Laminar organization of cellular microcircuits modulating human interictal epileptiform discharges.
Journal: Nature neuroscience
In common: seaborn, scikit-learn, pandas, 3 other tools, epilepsy, author Jonathan K Kleen
[6] doi:10.1038/s42003-026-10957-8 [code]
Brain defence by the extracellular matrix protein Cochlin.
Journal: Communications biology
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools
[7] doi:10.1038/s41467-026-76837-1 [code]
Drug screen and machine learning predict neuroprotective agents in a preclinical human model of childhood dementia.
Journal: Nature communications
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools
[8] doi:10.1038/s41467-026-74358-5 [code]
Brain-inspired spatial intelligence for embodied agents.
Journal: Nature communications
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools
[9] doi:10.7554/elife.107933 [code]
Modality-agnostic decoding of vision and language from fMRI.
Journal: eLife
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools
[10] doi:10.1002/hbm.70469 [code]
VarCoNet: A Variability-Aware Self-Supervised Framework for Functional Connectome Extraction From Resting-State fMRI.
Journal: Human brain mapping
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.