Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations.
14 blocks of code here have a permanent identifier (oscr:b:…), listed with the matches below: citable for good, with a page of its own. Select a match and press Cite to copy it. What a block identifier is.
The 16 matches
- [1] § Methods › Model training and evaluation ↔ tangle_tracer/nft_datasets.py, lines 33 to 82 · score 0.87 · Gaussian blurring, vertical flips, affine, brightness, horizontal, hue · block
oscr:b:MYYVNZWA2CFEZ-S· Report a mismatch - [2] § Methods › Point annotation to segmentation masks ↔ tangle_tracer/annot_conversion/ROIAnnotation.py, lines 306 to 370 · score 0.81 · Otsu thresholded, binary mask, scikit-image, rgb2hed, isolating, DAB · block
oscr:b:P3770EKRZ8Z82-Y· Report a mismatch - [3] § Methods › Point annotation to segmentation masks ↔ tangle_tracer/annot_conversion/WSIAnnotation.py, lines 436 to 500 · score 0.80 · Otsu thresholded, binary mask, scikit-image, rgb2hed, isolating, DAB · block
oscr:b:VA39WQBXQC026-6· Report a mismatch - [4] § Methods › Model training and evaluation ↔ tangle_tracer/modules.py, lines 13 to 38 · score 0.73 · Tversky loss, pre trained, encoders, UNet, beta, ResNet50 · block
oscr:b:PQH820REAQPT7-P· Report a mismatch - [5] § Methods › Comparing model predictions with AT8 burden and semi-quantitative scores ↔ tangle_tracer/annot_conversion/ROIAnnotation.py, lines 306 to 370 · score 0.73 · HED channels, Otsu thresholding, scikit-image, RGB, DAB, binarized · block
oscr:b:P3770EKRZ8Z82-Y· Report a mismatch - [6] § Methods › Comparing model predictions with AT8 burden and semi-quantitative scores ↔ tangle_tracer/annot_conversion/WSIAnnotation.py, lines 436 to 500 · score 0.73 · HED channels, Otsu thresholding, scikit-image, RGB, DAB, binarized · block
oscr:b:VA39WQBXQC026-6· Report a mismatch - [7] § Methods › Model training and evaluation ↔ tangle_tracer/train.py, lines 78 to 121 · score 0.72 · wandb sweep, weight decay, momentum, Tversky, alpha, epoch · block
oscr:b:5N6TBSEZJEMC7-S· Report a mismatch - [8] § Methods › Point annotation to segmentation masks ↔ tangle_tracer/annot_conversion/ROIAnnotation.py, lines 241 to 303 · score 0.70 · largest blob, center bias, scikit-image, background, skimage, empty · block
oscr:b:CH768QYFC67PN-8· Report a mismatch - [9] § Methods › Point annotation to segmentation masks ↔ tangle_tracer/annot_conversion/WSIAnnotation.py, lines 378 to 433 · score 0.70 · largest blob, center bias, scikit-image, background, skimage, empty · block
oscr:b:0DBD4DM0Y80JP-M· Report a mismatch - [10] § Methods › Object detection: training an object detection model from bootstrapped bounding boxes ↔ tangle_tracer/object_detection/train_yolo.py, lines 6 to 45 · score 0.68 · model.tune, hyperparameter search, Ultralytics, optimized, YOLOv8, train · block
oscr:b:MY2N3YB95EFXH-4· Report a mismatch - [11] § Methods › Datatype conversion ↔ tangle_tracer/nft_datasets.py, lines 86 to 151 · score 0.67 · 1–343, temporal at8, s1, scenes, space, shuffling · block
oscr:b:03S89815JZYKQ-D· Report a mismatch - [12] § Methods › Datatype conversion ↔ tangle_tracer/annot_conversion/czi_to_zarr.py, lines 68 to 123 · score 0.67 · multiple scenes, zstd, clevel, Blosc, compression, chunk · block
oscr:b:N3KW5VBRZPJQH-1· Report a mismatch - [13] § Methods › Model training and evaluation ↔ tangle_tracer/train.py, lines 14 to 74 · score 0.59 · ImageNet normalized, monitored, Tversky, ResNet50, hyperparameter, alpha · block
oscr:b:20R8C7H05TV7A-I· Report a mismatch - [14] § Methods › Model training and evaluation ↔ tangle_tracer/nft_datasets.py, lines 573 to 656 · score 0.59 · PyTorch, static, stride, inference, empty, rows · block
oscr:b:1E5Q3CQG29BEE-G· Report a mismatch - [15] § Methods › Metric choices ↔ tangle_tracer/modules.py, lines 71 to 145 · score 0.53 · AUPRC, AUROC, intersection, recall, aggregate, union · block
oscr:b:CFPD027JT2EY8-R· Report a mismatch - [16] § Methods › Rotated ROI correction ↔ tangle_tracer/annot_conversion/WSIAnnotation.py, lines 266 to 304 · score 0.53 · corrected ROI, inscribing, rotated, crop, disk, slicing · block
oscr:b:JZDRM2T8D94NV-I· Report a mismatch
Paper
Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
Python · 654 lines · 27 KB · MIT · 4 matches
- import numpy as np
- import zarr
- import cv2
- from tqdm import tqdm
- import os, shutil, sys
- from pathlib import Path
- from matplotlib import pyplot as plt
- from matplotlib.colors import ListedColormap
- import matplotlib as mpl
- import zarr
- from numcodecs import Blosc
- import pandas as pd
- import skimage
- import pickle
- from tangle_tracer.annot_conversion.cz_utils import CzParser
- from tangle_tracer.annot_conversion.czi_to_zarr import get_section
- # from cz_utils import CzParser
- # from zarr_utils import get_section #these imports cause pytest to glitch?
- #create a WSIAnnotation class
- class WSIAnnotation():
- """
- Goal: Return an info_dict that can be used to represent bounding boxes within the image (WSI or rectangles?)
- """
- def __init__(self, img_path, annot_path, scale = 1.0, save = True, overwrite = True,
- save_path = '/scratch/sghandian/projects/nft/model_input2/',
- reannotations = '/srv/home/lianeb/reannotations.csv'):
- """
- Initialize WSIAnnotation.
- """
- try:
- self.reannotations = pd.read_csv(reannotations)
- except FileNotFoundError:
- print("Default reannotation file not found. Either pass in None for reannotations parameter or change path.")
- self.reannotations = []
- self.scale = scale
- img_path = Path(img_path)
- cz_annot_path = self.__create_annot_path(annot_path)
- self.annot_data = CzParser(cz_annot_path).get_regions()
- self.wsi_name = img_path.as_posix().split('/')[-1].split('.')[0]
- self.wsi_img = zarr.open(img_path, mode = 'r', dtype = 'i4')
- #define bbox size
- self.bbox_size = int(400 * self.scale)
- #define input tile size for model, pad ROI by this much + jitter amount
- self.input_tile_size = int(1024 * self.scale)
- #how much to vary random sampling coords of positive class
- self.jitter_amount = self.input_tile_size // 2
- #calculate total padding size
- self.padding_size = self.input_tile_size + self.jitter_amount
- #need save_path for zarr dump of corrected ROIs
- self.save_path = save_path
- self.chunk_size = 1000 #used for zarr chunks
- self.overwrite = overwrite
- try:
- self.img_regions = self._get_img_regions(self.annot_data['regions'])
- self.bboxes = self._get_img_bboxes(self.img_regions)
- self.masks = self._draw_all_masks(biggest = False)
- except FileNotFoundError:
- print(f'No annotation data found for {self.wsi_name}. \
- Returning WSIAnnot with reference to zarr image only.')
- if save:
- self.save_annot()
- def __create_annot_path(self, annot_path):
- basename = os.path.basename(annot_path)
- if '_s' in basename:
- name = "_".join(basename.split('_')[:-1])
- annot_path = Path("/".join(annot_path.as_posix().split('/')[:-1]),
- name).with_suffix('.cz')
- return annot_path
- def __get_nft_bboxes(self, rectangle):
- """
- Creates bbox DIMENSIONS around each NFT in a given ROI. Filters out NFTs
- that are not within this ROI.
- Input:
- rectangle (str) - Index to self.annot_data['regions'] which contains
- metadata specific to this ROI within the larger WSI.
- Returns:
- bboxes (dict) - Maps an NFT (str) to its bbox dimensions (dict).
- """
- #define nft and rectangle metadata
- rect_annots = self.annot_data['regions'][rectangle]
- offset = (rect_annots['Left'] * self.scale,
- rect_annots['Top'] * self.scale)
- angle = -(rect_annots['Rotation']) #use negative version
- rect_width, rect_height = rect_annots['Width'] * self.scale, rect_annots['Height'] * self.scale
- rect_center = np.int32([rect_width//2, rect_height//2])
- #use local coords if provided
- if 'nfts' in rect_annots:
- nft_annots = rect_annots['nfts']
- else:
- nft_annots = self.annot_data['nfts']
- #define new point annotations
- num_nfts = len(nft_annots) #number of original nfts
- #define rotation matrix M
- M = self.get_rotation_matrix(angle)
- def process_annots_into_bboxes(annot_dict, bbox_size, rect_width, rect_height, scale = 1.0):
- #bbox border in each direction from center
- bboxes = {}
- border = int(bbox_size // 2)
- for nft, coord in annot_dict.items():
- x,y = int(coord[0] * scale), int(coord[1] * scale)
- dims = {
- 'xmin': int(x - border),
- 'ymin': int(y - border),
- 'xmax': int(x + border),
- 'ymax': int(y + border),
- 'width': int(border * 2),
- 'height': int(border * 2), #can likely get rid of width and height
- }
- #check to make sure the bbox is in this rectangle
- if all(int(x) > - border for x in dims.values()): #filtered out by >0 condition
- if (dims['xmax'] <= rect_width + border) and (dims['ymax'] <= rect_height + border): #could have problems w/ border and edge NFTs
- bboxes[nft] = dims
- return bboxes
- def apply_rotation_to_bboxes(bboxes, bbox_size, rect_width, rect_height, offset, scale = 1.0):
- #check each global NFT annot for existence in this ROI (after rotation)
- border = int(bbox_size // 2)
- for nft, coord in bboxes.items():
- x, y = int(coord[0] * self.scale), int(coord[1] * self.scale)
- #convert to local coordinates
- xy = np.int32([x - offset[0], y - offset[1]])
- #shift coords about center of rectangle
- xy -= rect_center
- #use rotation matrix, order matters!
- if 'nfts' in rect_annots:
- x, y = np.int32(M @ xy) + np.flip(rect_center)
- #may need to add a check based on final orientation
- else:
- #rect_center to correct shift after rotation, not for local
- #shift coordinates back from origin for global coords!
- x, y = np.int32(M @ xy) + rect_center
- dims = {
- 'xmin': int(x - border),
- 'ymin': int(y - border),
- 'xmax': int(x + border),
- 'ymax': int(y + border),
- 'width': int(border * 2),
- 'height': int(border * 2), #can likely get rid of width and height
- }
- #check to make sure the bbox is in this rectangle
- if all(int(x) > - border for x in dims.values()): #filtered out by >0 condition
- if (dims['xmax'] <= rect_width + border) and (dims['ymax'] <= rect_height + border): #could have problems w/ border and edge NFTs
- bboxes[nft] = dims
- return bboxes
- #post-process these boxes by adding the pad_amount to all of them!
- def modify_all_dims(bboxes, pad_amount = self.padding_size):
- def modify_dims(bbox_dims, pad_amount = pad_amount):
- bbox_dims = bbox_dims.copy()
- for dim, val in bbox_dims.items():
- if dim in ['width', 'height']:
- continue
- bbox_dims[dim] = val + pad_amount
- return bbox_dims
- #copy bboxes and apply modify_dims to each bbox
- bboxes = bboxes.copy()
- for annot, bbox_dim in bboxes.items():
- bboxes[annot] = modify_dims(bbox_dim)
- return bboxes
- #convert raw NFT annotations from WSI-level into bboxes
- bboxes = process_annots_into_bboxes(nft_annots, self.bbox_size, rect_width, rect_height, self.scale)
- bboxes = apply_rotation_to_bboxes(bboxes, self.bbox_size, rect_width, rect_height, offset, self.scale)
- bboxes = modify_all_dims(bboxes, pad_amount = self.padding_size)
- if len(self.reannotations) > 0:
- df = self.reannotations
- reannot_roi = df[(df['ROI'] == int(rectangle.split('_')[-1])) & (df['WSI'] == self.wsi_name)]
- nft_reannots = reannot_roi['ROI_coordinates'].apply(eval).to_dict()
- new_keys = ['annotation_' + str(k+num_nfts) for k in nft_reannots.keys()]#nft_reannots.keys()]
- new_vals = [list(v) for v in nft_reannots.values()]
- nft_reannots = dict(zip(new_keys, new_vals))
- #no need to apply rotation to these as they were annotated at the ROI-level (pre-rotated)
- new_bboxes = process_annots_into_bboxes(nft_reannots, self.bbox_size, rect_width, rect_height, self.scale)
- new_bboxes = modify_all_dims(new_bboxes, pad_amount = 0)
- bboxes = {**bboxes, **new_bboxes}
- return bboxes
- def _transform_rect(self, metadata):
- #define and scale dimensions
- left, top = int(metadata['Left'] * self.scale), int(metadata['Top'] * self.scale)
- width, height = int(metadata['Width'] * self.scale), int(metadata['Height'] * self.scale)
- border, angle = 0, -metadata['Rotation'] #must use negative rot
- #get a path to a roi jpeg if it exists
- roi_path = metadata.get('Path', None)
- #calculate rotation M for given angle
- rotation_M = self.get_rotation_matrix(angle)
- #define points w/ respect to origin
- pt_A = [0, 0]
- pt_B = [0 + width, 0]
- pt_C = [0 + width, 0 + height]
- pt_D = [0, 0 + height]
- pts = np.int32([pt_A, pt_B, pt_C, pt_D])
- #shift to center of rectangle
- center = np.int32([[width//2, height//2]] * 4)
- pts = pts - center
- #define shifting params (border + offset dims)
- shift = np.float32([[border + left, border + top]] * 4) + center
- #create rotation matrix and input points
- in_pts = np.float32(pts @ rotation_M) + shift
- #define strict output points (height/width of rectangle)
- out_pts = np.float32([[0, 0],
- [width - 1, 0],
- [width - 1, height - 1],
- [0, height - 1]])
- #before doing transform, feed in a slice that
- #is cropped around the minimal region
- minCoords = np.flip(np.int32(in_pts.min(axis = 0)))
- maxCoords = np.flip(np.int32(in_pts.max(axis = 0)))
- tileSize = maxCoords - minCoords
- if not roi_path and self.wsi_img:
- img_slice = get_section(self.wsi_img, minCoords,
- tile_size = (tileSize))
- else:
- #load directory from jpeg if there's no WSI but there are ROIs
- try:
- img_slice = mpl.image.imread(roi_path)
- except FileNotFoundError:
- print('Path to ROI not found: {roi_path}. Fix path to ROI in metadata if no WSI exists.')
- #convert in_pts to local coordinates of img_slice
- in_pts -= np.flip(minCoords)
- #compute the perspective transform M
- M = cv2.getPerspectiveTransform(np.float32(in_pts),
- np.float32(out_pts))
- #warp the slice with the rotation matrix
- out = skimage.transform.warp(img_slice, np.linalg.inv(M),
- output_shape=(height, width))
- return out
- def _get_img_regions(self, rectangles):
- """
- Takes in the ROI annotations and generates a nested dictionary mapping rectangles to the ROI image,
- corrected for rotation, as well as the constitutent bbox dimensions for that specific rectangle.
- Input:
- rectangles (dict): Dictionary mapping rectangle name to global coordinates and rotation.
- Returns:
- img_slices (dict): Dictionary mapping rectangle name to corrected ROI view in np.array format,
- and coordinates of bboxes.
- """
- img_slices = {}
- #can potentially choose the relevant region first,
- for rectangle, metadata in tqdm(rectangles.items()):
- #uses self.wsi_img and coords to slice minimal inscribed img,
- #then rotates and returns the correct view of this cropped img
- img_slice = self._transform_rect(metadata)
- #pad array by padding amount on each side,
- #MIGHT WANT THIS TO BE TILE SIZE !!! 1024px
- padded_img = np.pad(img_slice, ((self.padding_size, self.padding_size),
- (self.padding_size, self.padding_size), (0, 0)),
- mode = 'constant', constant_values = 1)
- #convert img_slice to zarr file on disk?
- os.makedirs(Path(self.save_path, 'images', self.wsi_name), exist_ok=True)
- wsi_path = Path(self.save_path,'images', self.wsi_name, rectangle)
- zarr_arr = self.__dump_to_zarr(padded_img, wsi_path)
- #uses rectangle coords + angle to generate bounding boxes
- bboxes = self.__get_nft_bboxes(rectangle)
- #save to dictionary
- img_slices[rectangle] = {'img': zarr_arr, 'bboxes': bboxes}
- return img_slices
- def __dump_to_zarr(self, img: np.array, path: Path):
- # compressor = Blosc(cname='lz4', clevel=1, shuffle=1)
- compressor = Blosc(cname='zstd',clevel=5, shuffle=Blosc.BITSHUFFLE)
- synchronizer = zarr.ProcessSynchronizer(path.with_suffix('.sync'))
- store = zarr.DirectoryStore(path)
- if self.overwrite:
- try:
- shutil.rmtree(path)
- except FileNotFoundError:
- print('Cannot overwrite ROI that does not exist. Creating new zarr array.')
- try:
- z = zarr.array(img,
- store=store,
- chunks = (self.chunk_size, self.chunk_size),
- write_empty_chunks=False,
- compressor = compressor,
- synchronizer = synchronizer)
- except zarr.errors.ContainsArrayError:
- print(f'Skipping {path} since it already exists.')
- z = zarr.open(path, mode = 'r')
- return z
- def _load_bbox(self, rectangle_img, bbox_object):
- """
- Takes in an annotation for a bounding box and retuns a slice of the rectangle image.
- Input:
- rectangle_img (np.array): Slice of WSI in np format
- bbox_object (dict): Dictionary {annotation_# : xmin, ymin, xmax, ymax, width, height}
- scale (float) : Scale at which the image is produced. Will scale bbox coordinates appropriately.
- Returns:
- box_slice (np.array): Slice of the image contained in the bounding box, padded to 400px if necessary.
- """
- left = int(bbox_object['xmin'])
- top = int(bbox_object['ymin'])
- width = int(bbox_object['width'])
- height = int(bbox_object['height'])
- #add edge case handling via checking coordinates, altering indexing and padding
- box_slice = rectangle_img[slice(np.max([0, top]), top + height),
- slice(np.max([0, left]), left + width), :].copy()
- return box_slice
- def _get_img_bboxes(self, img_regions):
- """
- Takes in the dictionary of rectangles mapped to its image, as well as its constituent bboxes dimensions,
- and produces all associated bbox images in a dictionary mapping annotation name to image.
- Input:
- img (zarr.array): WSI in Zarr format
- img_regions (dict): Dictionary for a single WSI mapping each annotated rectangle name to another dictionary,
- containing image region and its associated annotated NFTs in bounding box format.
- Example:
- img_regions['rectangle_0'] = {'img': img_slice, 'bboxes': bboxes}
- Returns:
- bbox_imgs (dict): Dictionary mapping an NFT bounding box annotation to an image slice.
- Example:
- bbox_imgs['annotation_0'] = bbox
- """
- bbox_imgs = {}
- for rectangle in img_regions:
- for bbox, dims in img_regions[rectangle]['bboxes'].items():
- bbox_imgs[bbox] = self._load_bbox(img_regions[rectangle]['img'], dims)
- return bbox_imgs
- def _create_blobs(self, mask, threshold=1500, max_distance = 125):
- """
- Takes in a binarized bbox image and generates a mask representing a single NFT within it.
- """
- labels = skimage.measure.label(mask, connectivity=2, background=0)
- out_mask = np.zeros(mask.shape, dtype='uint8')
- sizes = {}
- # loop over the unique components
- for label in np.unique(labels):
- # if this is the background label, ignore it
- if label == 0:
- continue
- # otherwise, construct the label mask and count the
- # number of pixels
- labelMask = np.zeros(mask.shape, dtype="uint8")
- labelMask[labels == label] = 255
- numPixels = cv2.countNonZero(labelMask)
- # if the number of pixels in the component is sufficiently
- # large, then add it to our dictionary mapping labels to sizes
- if numPixels > threshold*(self.scale**2):
- #enforce center bias
- blob_center = np.float32([np.average(indices) for indices in np.where(labelMask == 255)])
- img_center = np.float32([200*self.scale, 200*self.scale])
- if np.linalg.norm(blob_center - img_center) < max_distance*self.scale:
- sizes[label] = numPixels
- #get the largest blob
- try:
- maxLabel = max(sizes, key = sizes.get)
- except ValueError: #nothing found
- #---------------- consider adding the next smallest blob!!! -------------
- return out_mask, out_mask
- #pop the max label
- maxPixel = sizes.pop(maxLabel)
- #generate a mask with the largest label
- biggest_mask = np.zeros(mask.shape, dtype="uint8")
- biggest_mask[labels == maxLabel] = 255
- # biggest_mask_size = cv2.countNonZero(biggest_mask)
- out_mask = cv2.add(out_mask, biggest_mask)
- #get the biggest blobs within X% of max size
- percentage_threshold = 0.50
- #loop through all sufficiently large blobs and get sizes
- for label, pixelSize in sizes.items():
- #check if they're large enough to add to an output mask
- if (pixelSize / maxPixel) > percentage_threshold:
- #create an empty mask, then assign 1s to labeled region
- labelMask = np.zeros(mask.shape, dtype="uint8")
- labelMask[labels == label] = 255
- #add label mask to output mask
- out_mask = cv2.add(out_mask, labelMask)
- return out_mask, biggest_mask
- #try otsu thresholding
- def _draw_mask(self, image, show_img = False, overlay = False):
- """
- Takes in an image and converts it from RGB to HED channels. The image is then normalized and
- converted to uint8, then the DAB channel is isolated and is binarized and thresholded via Otsu.
- Morphological operations then preprocess the image, then it's passed to _create_blobs() to generate
- the segmented mask for the image. Can use show_img and overlay flags to visualize results.
- Input:
- image (np.array): RGB image representing a zoomed-in NFT image from an ROI.
- show_img (bool): Flag to display image in notebook
- overlay (bool): Flag to overlay the mask onto the image. RETURNS the overlaid mask!
- Retuns:
- mask, biggest_mask (tuple): Mask images that represent the most central mask and the biggest masked
- region within the bbox.
- """
- og_image = np.array(skimage.color.rgb2hed(image).copy())
- normed_image = None
- normed_image = cv2.normalize(og_image, normed_image, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8U)
- #select DAB color channel
- try:
- DAB = normed_image[:, :, 2]
- except TypeError:
- print(f"OG image shape {og_image.shape} and image: {og_image}")
- return (None, None)
- #create binary mask
- thresh = cv2.threshold(DAB, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
- #use cross as kernel
- kernel = cv2.getStructuringElement(cv2.MORPH_CROSS, (3, 3))
- # Apply morphological closing, then opening operations
- closing = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
- # for _ in range(10):
- # closing = cv2.morphologyEx(closing, cv2.MORPH_CLOSE, kernel)
- # opening = cv2.morphologyEx(closing, cv2.MORPH_OPEN, kernel)
- #create blobs and their masks
- mask, biggest_mask = self._create_blobs(closing, threshold = 1000 * self.scale)
- if overlay:
- # Choose colormap
- cmap = plt.get_cmap('cool')
- # Get the colormap colors
- my_cmap = cmap(np.arange(cmap.N))
- # Set alpha to 0 for non values of 1
- my_cmap[:,-1][:255] = 0
- # Create new colormap
- my_cmap = ListedColormap(my_cmap)
- #apply colormap to mask
- mask = my_cmap(mask)
- #add an extra dimension to image
- image = cv2.cvtColor(np.float32(image), cv2.COLOR_RGB2RGBA)
- if show_img:
- # fig = plt.figure()
- plt.imshow(image)
- plt.imshow(mask, cmap = 'binary')
- return mask, biggest_mask
- def _draw_all_masks(self, biggest = False):
- """
- Iterates through self.bboxes to retrieve 400x400px images for segmentation. Generates mask(s)
- based on strategy defined in _draw_mask(). Returns dictionary mapping NFT annotation name to
- a single grayscale image.
- """
- masks = {}
- for (nft, bbox) in self.bboxes.items():
- mask, biggest_mask = self._draw_mask(bbox, overlay = True)
- if biggest:
- masks[nft] = biggest_mask
- else:
- masks[nft] = mask
- return masks
- #generate a rotation matrix using degrees
- def get_rotation_matrix(self, angle):
- """
- Generate a 2d rotation matrix with an input angle, defined in degrees.
- """
- angle = np.deg2rad(angle)
- R = np.array([[np.cos(angle), -np.sin(angle)],
- [np.sin(angle), np.cos(angle)]])
- return R
- def save_annot(self):
- """
- Dumps the WSIAnnot object at the default save location for the specific scale at which
- the original WSI was loaded in (self.scale).
- Can be loaded again with WSIAnnotation.load_annot(WSI_NAME).
- """
- save_path = Path(self.save_path,'wsiAnnots', f'scale_{self.scale}')
- #create a directory if needed
- os.makedirs(save_path, exist_ok=True)
- pickle_path = Path(save_path, f"{self.wsi_name}.pickle")
- #remove pickle file/directory if it does exist
- if os.path.isfile(pickle_path) or os.path.islink(pickle_path):
- os.remove(pickle_path)
- elif os.path.isdir(pickle_path):
- shutil.rmtree(pickle_path)
- #dump pickle file
- with open(Path(save_path, f"{self.wsi_name}.pickle"), "wb") as file_to_store:
- pickle.dump(self, file_to_store)
- #----------------------------Utility Functions----------------------------#
- def plot_bboxes(wsiAnnot, num_imgs = 12, overlay = True):
- bbox_imgs = wsiAnnot.bboxes
- masks = wsiAnnot.masks
- num_imgs = min([num_imgs, len(bbox_imgs.keys())])
- fig, ax = plt.subplots(num_imgs // 4, 4, figsize=(16, num_imgs), sharex = True, sharey = True)
- ax = ax.flatten()
- max_imgs = (num_imgs // 4) * 4
- for i, (nft, bbox) in enumerate(bbox_imgs.items()):
- if i > max_imgs - 1 or i >= len(bbox_imgs.keys()) - 1:
- break
- ax[i].imshow(bbox)
- if overlay:
- ax[i].imshow(masks[nft])
- ax[i].set_title(nft)
- fig.suptitle(f'NFT Bounding Boxes - {wsiAnnot.wsi_name}')
- fig.tight_layout(rect=[0, 0.02, 1, 0.98])
- def load_annot(wsiAnnot_name, load_path = '/scratch/sghandian/nft/model_input_v2/wsiAnnots/', scale = 1.0):
- out_path = Path(load_path, f'scale_{scale}', f"{wsiAnnot_name}.pickle")
- with open(out_path, "rb") as file_to_read:
- try:
- loaded_object = pickle.load(file_to_read)
- except ModuleNotFoundError: #handle unpickling error (pickled with abs import instead of current relative import)
- file_to_read.seek(0)
- curr_dir = os.path.realpath(os.path.dirname(__file__))
- sys.path.insert(0, curr_dir)
- loaded_object = pickle.load(file_to_read)
- return loaded_object
- #----------------------------Utility Functions----------------------------#
- def main(args):
- SCALE = args.scale
- ANNOT_PATH = Path(args.annot_path)
- ZARR_PATH = Path(args.img_path, f'scale_{SCALE}/')
- SAVE_PATH = Path(args.save_path)
- #choose which wsi to create, can use indices for ease but only need to use one of the following
- WSI_NAME = args.wsi
- idx = args.idx
- filenames = os.listdir(ZARR_PATH)
- #if wsi_name passed in or idx given, load that one specifically and save it to disk
- if WSI_NAME or idx is not None:
- if WSI_NAME:
- file = WSI_NAME + '.zarr'
- else:
- file = filenames[idx]
- print(f"WSI name: {file}")
- imgPath = Path(ZARR_PATH, file)
- annotPath = Path(ANNOT_PATH, Path(file).with_suffix('.cz'))
- wsiAnnot_single = WSIAnnotation(imgPath, annotPath, SCALE, save = True,
- save_path=SAVE_PATH)
- return wsiAnnot_single
- #create and save each WSIAnnotation sequentially, unlike parallel_annots.py
- for file in filenames:
- imgPath = Path(ZARR_PATH, file)
- annotPath = Path(ANNOT_PATH, Path(file).with_suffix('.cz'))
- if not annotPath.isfile():
- print("""Skipping {file} as {annotPath} annotation does not exist.
- Check whether annot_path arg is correct if this image does have an annotation.""")
- continue #skip files that do not have .cz associated
- wsiAnnot = WSIAnnotation(imgPath, annotPath, SCALE, save = True,
- save_path=SAVE_PATH)
- if __name__ == '__main__':
- from argparse import ArgumentParser
- #add an argparser
- parser = ArgumentParser()
- parser.add_argument("--img_path", type=str, required=True) #need to pass one of these args in!
- parser.add_argument("--annot_path", type=str, required=True) #need to pass one of these args in!
- parser.add_argument("--wsi", type=str) #need to pass one of these args in!
- parser.add_argument("--idx", type=int) #need to pass one of these args in!
- parser.add_argument("--scale", type=float, default = 1.0)
- parser.add_argument("--save_path", type=str, default='/scratch/sghandian/projects/nft/')
- args = parser.parse_args()
- #run on all images
- main(args)
- """
- Example Usage:
- python annot_conversion/point_to_mask.py \
- --img_path='/scratch/sghandian/nft/raw_data/wsis/zarr/' \
- --annot_path='/scratch/sghandian/nft/raw_data/annotations/updated_annots/' \
- --idx=0 --scale=0.1 --save_path='../data/'
- """
WSIAnnotation.py at commit 8778679, under MIT · at the source
Overview
- Institute for Neurodegenerative Diseases, University of California, San Francisco,San Francisco, CA 94158 USA
- Bakar Computational Health Sciences Institute, University of California,San Francisco, CA 94158 USA
- Department of Pharmaceutical Chemistry, University of California, San Francisco,San Francisco, CA 94158 USA
- Department of Bioengineering and Therapeutic Sciences, University of California, San Francisco,San Francisco, CA 94158 USA
- Department of Pathology and Laboratory Medicine, School of Medicine, University of California, Davis,Sacramento, CA 95817 USA
- Department of Computer Science, University of California, Davis,Davis, CA 95616 USA
- Robust and Ubiquitous Networking (RUbiNet) Lab, University of California, Davis,Davis, CA 95616 USA
- Division of Biostatistics, Department of Public Health Sciences, University of California Davis,Davis, CA USA
- Department of Neurology, School of Medicine, Alzheimer’s Disease Research Center, University of California Davis,Sacramento, CA USA
- Department of Neurosciences, University of California San Diego,La Jolla, San Diego, CA USA
- Department of Neurology, Taub Institute for Research On Alzheimer’s Disease and Aging Brain, Columbia University Medical Center,New York, NY USA
Abstract
Accumulation of abnormal tau protein into neurofibrillary tangles (NFTs) is a pathologic hallmark of Alzheimer disease (AD). Accurate detection of NFTs in tissue samples can reveal relationships with clinical, demographic, and genetic features through deep phenotyping. However, expert manual analysis is time-consuming, subject to observer variability, and cannot handle the data amounts generated by modern imaging. We present a scalable, open-source, deep-learning approach to quantify NFT burden in digital whole slide images (WSIs) of post-mortem human brain tissue. To achieve this, we developed a method to generate detailed NFT boundaries directly from single-point-per-NFT annotations. We then trained a semantic segmentation model on 45 annotated 2400 μm by 1200 μm regions of interest (ROIs) selected from 15 unique temporal cortex WSIs of AD cases from three institutions (University of California (UC)-Davis, UC-San Diego, and Columbia University). Segmenting NFTs at the single-pixel level, the model achieved an area under the receiver operating characteristic of 0.832 and an F1 of 0.527 (196-fold over random) on a held-out test set of 664 NFTs from 20 ROIs (7 WSIs). We compared this to deep object detection, which achieved comparable but coarser-grained performance that was 60% faster. The segmentation and object detection models correlated well with expert semi-quantitative scores at the whole-slide level (Spearman’s rho ρ = 0.654 (p = 6.50e-5) and ρ = 0.513 (p = 3.18e-3), respectively). We openly release this multi-institution deep-learning pipeline to provide detailed NFT spatial distribution and morphology analysis capability at a scale otherwise infeasible by manual assessment.
Supplementary Information: The online version contains supplementary material available at https://
Reproduced under the paper's license (CC BY), from the paper cited above.
Repository
Its files are read in the Code ↔ Paper reader above, with 16 matches between paragraphs and lines of code.
keiserlab/tangle-tracer
877867971521646aefef9124a9b2b44ff775a0d0, 11 December 2025Availability: 1 check, the latest on 27 September 2026: the link answers
- 27 September 2026: the link answers
31 files
- setup.py, Python, 6 lines
- tangle_tracer/
__init__.py , Python, 9 lines - tangle_tracer/
annot_conversion/ , Python, 409 lines, 3 matchesROIAnnotation.py - tangle_tracer/
annot_conversion/ , Python, 191 linesROIParser.py - tangle_tracer/
annot_conversion/ , Python, 654 lines, 4 matchesWSIAnnotation.py - tangle_tracer/
annot_conversion/ , Python, 1 line__init__.py - tangle_tracer/
annot_conversion/ , Python, 61 linescz_utils.py - tangle_tracer/
annot_conversion/ , Python, 151 lines, 1 matchczi_to_zarr.py - tangle_tracer/
annot_conversion/ , Python, 205 linesmask_stitching.py - tangle_tracer/
annot_conversion/ , Python, 102 linesparallel_annots.py - tangle_tracer/
inference.py , Python, 194 lines - tangle_tracer/
modules.py , Python, 387 lines, 2 matches - tangle_tracer/
nft_datasets.py , Python, 871 lines, 3 matches - tangle_tracer/
object_detection/ , Python, 1 line__init__.py - tangle_tracer/
object_detection/ , Python, 85 linescreate_tiles_yolo.py - tangle_tracer/
object_detection/ , Python, 72 linesmodules_yolo.py - tangle_tracer/
object_detection/ , Python, 204 linesroi_inference_yolo.py - tangle_tracer/
object_detection/ , Python, 68 lines, 1 matchtrain_yolo.py - tangle_tracer/
object_detection/ , Python, 185 lineswsi_inference_yolo.py - tangle_tracer/
object_detection/ , Python, 355 linesyolo_datasets.py - tangle_tracer/
scoring/ , Python, 39 linescalc_tissue_area.py - tangle_tracer/
scoring/ , Python, 71 linescount_nfts.py - tangle_tracer/
train.py , Python, 124 lines, 2 matches - tests/
__init__.py , Python, 1 line - tests/
metric_check.py , Python, 73 lines - tests/
test_NFTDetector.py , Python, 30 lines - tests/
test_WSIAnnotation.py , Python, 63 lines - tests/
test_yolo_datamodule.py , Python, 1 line - tests/
test_yolo_module.py , Python, 29 lines - LICENSE, License, 21 lines
- README.md, Text, 64 lines
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 29 scripts, each with its path and the digest of its content;
- 16 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Datasets cited
- biostudies:S-BIAD1165, at BioStudies; found in “Data availability”
Data availability
The images (rotated and cropped regions of interest and their segmentation masks) used to train and evaluate all machine learning models and data created and used in this project, including results, are available at https://
Reproduced under the paper's license (CC BY), from the paper cited above.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 13 authors, 18 keywords, 7 MeSH terms, 2 funders, 47 references, 1 RRID.
Cite
This paper
Ghandian, S., Albarghouthi, L., Nava, K., Sharma, S. R. R., Minaud, L., Beckett, L., Saito, N., DeCarli, C., Rissman, R. A., Teich, A. F., Jin, L.-W., Dugger, B. N., & Keiser, M. J. (2026). Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations. Scientific reports, 16(1), 27095. https://
BibTeX
@article{ghandian2026lea
author = {Ghandian, Sina and Albarghouthi, Liane and Nava, Kiana and Sharma, Shivam R. Rai and Minaud, Lise and Beckett, Laurel and Saito, Naomi and DeCarli, Charles and Rissman, Robert A. and Teich, Andrew F. and Jin, Lee-Way and Dugger, Brittany N. and Keiser, Michael J.},
title = {{Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations}},
journal = {Scientific reports},
year = {2026},
month = aug,
volume = {16},
number = {1},
pages = {27095},
publisher = {Nature Publishing Group},
issn = {2045-2322},
doi = {10.1038/
url = {https://
pmid = {42665582},
pmcid = {PMC13524952}
}
RIS
TY - JOUR
AU - Ghandian, Sina
AU - Albarghouthi, Liane
AU - Nava, Kiana
AU - Sharma, Shivam R. Rai
AU - Minaud, Lise
AU - Beckett, Laurel
AU - Saito, Naomi
AU - DeCarli, Charles
AU - Rissman, Robert A.
AU - Teich, Andrew F.
AU - Jin, Lee-Way
AU - Dugger, Brittany N.
AU - Keiser, Michael J.
TI - Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations
T2 - Scientific reports
J2 - Sci Rep
PY - 2026
DA - 2026/
VL - 16
IS - 1
SP - 27095
SN - 2045-2322
PB - Nature Publishing Group
DO - 10.1038/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1038/
"type": "article-journal",
"title": "Learning precise segmentation of neurofibrillary tangles from rapid manual point annotations",
"container-title": "Scientific reports",
"author": [
{
"family": "Ghandian",
"given": "Sina"
},
{
"family": "Albarghouthi",
"given": "Liane"
},
{
"family": "Nava",
"given": "Kiana"
},
{
"family": "Sharma",
"given": "Shivam R. Rai"
},
{
"family": "Minaud",
"given": "Lise"
},
{
"family": "Beckett",
"given": "Laurel"
},
{
"family": "Saito",
"given": "Naomi"
},
{
"family": "DeCarli",
"given": "Charles"
},
{
"family": "Rissman",
"given": "Robert A."
},
{
"family": "Teich",
"given": "Andrew F."
},
{
"family": "Jin",
"given": "Lee-Way"
},
{
"family": "Dugger",
"given": "Brittany N."
},
{
"family": "Keiser",
"given": "Michael J."
}
],
"container-title-short":
"volume": "16",
"issue": "1",
"page": "27095",
"DOI": "10.1038/
"PMID": "42665582",
"PMCID": "PMC13524952",
"ISSN": "2045-2322",
"publisher": "Nature Publishing Group",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
8,
28
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1186/s40478-024-01827-7 [code]
- Machine learning quantification of Amyloid-β deposits in the temporal lobe of 131 brain bank cases.Journal: Acta neuropathologica communicationsIn common: OpenCV, scikit-image, Pillow, 5 other tools, histology / microscopy, Alzheimer's / dementia, 11 references
- [2] doi:10.1093/jnen/nlaf152 [code]
- Clinical and pathologic correlations of machine learning quantification of Aβ deposits across 3 brain regions of decedents with Alzheimer disease.Journal: Journal of neuropathology and experimental neurologyIn common: OpenCV, scikit-image, Pillow, 5 other tools, Alzheimer's / dementia, 9 references
- [3] doi:10.1093/jnen/nlae120 [code]
- A machine learning approach to automate microinfarct and microhemorrhage screening in hematoxylin and eosin-stained human brain tissues.Journal: Journal of neuropathology and experimental neurologyIn common: OpenCV, scikit-image, Pillow, 5 other tools, histology / microscopy, Alzheimer's / dementia, 8 references
- [4] doi:10.1016/j.crmeth.2022.100191 [code]
- AI-enabled <i>in silico</
i> immunohistochemical characterization for Alzheimer's disease. Journal: Cell reports methodsIn common: OpenCV, scikit-image, Pillow, 4 other tools, histology / microscopy, Alzheimer's / dementia, 6 references - [5] doi:10.1186/s40478-021-01271-x [code]
- Deep learning reveals disease-specific signatures of white matter pathology in tauopathies.Journal: Acta neuropathologica communicationsIn common: OpenCV, scikit-image, Pillow, 4 other tools, histology / microscopy, Alzheimer's / dementia, 5 references
- [6] doi:10.1093/jnen/nlab122
- Advances in Deep Neuropathological Phenotyping of Alzheimer Disease: Past, Present, and Future.Journal: Journal of neuropathology and experimental neurologyIn common: histology / microscopy, Alzheimer's / dementia, 9 references
- [7] doi:10.1109/access.2022.3171927 [code]
- BrainSec: Automated Brain Tissue Segmentation Pipeline for Scalable Neuropathological Analysis.Journal: IEEE access : practical innovations, open solutionsIn common: OpenCV, scikit-image, Pillow, 5 other tools, histology / microscopy, methods / tools, Alzheimer's / dementia, 3 references
- [8] doi:10.17879/freeneuropathology-2025-6387 [code]
- Applying machine learning to assist in the morphometric assessment of brain arteriolosclerosis through automation.Journal: Free neuropathologyIn common: OpenCV, scikit-image, Pillow, 4 other tools, histology / microscopy, 4 references
- [9] doi:10.1007/s00401-024-02789-9 [code]
- Association of quantitative histopathology measurements with antemortem medial temporal lobe cortical thickness in the Alzheimer's disease continuum.Journal: Acta neuropathologicaIn common: PyTorch, NumPy, histology / microscopy, Alzheimer's / dementia, 6 references
- [10] doi:10.1162/imag.a.1079 [code]
- From fibers to cells: Fourier-based registration enables virtual Cresyl violet staining from 3D polarized light imaging.Journal: Imaging neuroscience (Cambridge, Mass.)In common: PyTorch Lightning, OpenCV, scikit-image, 6 other tools, histology / microscopy, 1 reference
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 1 repository of the authors' code, each at its verified commit and with its license, 29 scripts, and 16 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:268cee20bb79e268…
Add the badge to the code's README
If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.
Markdown
<!-- oscr-badge:start -->
[](https://openscicode.org/badge/doi/10.1038/s41598-026-61605-4)
<!-- oscr-badge:end -->reStructuredText
.. oscr-badge:start
.. image:: https://openscicode.org/badge/doi/10.1038/s41598-026-61605-4.svg
:target: https://openscicode.org/badge/doi/10.1038/s41598-026-61605-4
:alt: OSCR
.. oscr-badge:endHTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1038/s41598-026-61605-4"><img src="https://openscicode.org/badge/doi/10.1038/s41598-026-61605-4.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".
If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.
Report a mismatch between the code and the paper · Report a code error (signed in)
Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.
Report a reproduction failure (signed in)
Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
