OSCR

Large vision model framework for automated C. elegans analysis: From static morphometry to dynamic neural activity.

Code ↔ Paper

13 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 13 matches
  1. [1] § Materials and methods › Droplet swimming recording and analysis › Automated quantitative analysis of swimming behavior. ↔ droplet_swimming/4_shape_analysis.py, lines 525 to 619 · score 0.91 · power spectral density, Temporal frequencies, dominant frequencies, worm length, Welch, PSD · Report a mismatch
  2. [2] § Materials and methods › Droplet swimming recording and analysis › Automated quantitative analysis of swimming behavior. ↔ singleworm_tracking/3_shape_analysis.py, lines 804 to 943 · score 0.89 · power spectral density, dominant frequencies, worm length, Welch, PSD, Wave · Report a mismatch
  3. [3] § Materials and methods › Static images acquisition and analysis › Automated multi-worm morphological feature extraction from images. ↔ multiworm_feature_extraction/0_cutout_classifier.py, lines 18 to 49 · score 0.80 · random resized crops, horizontal flips, augmentation, PyTorch, validation, cutouts · Report a mismatch
  4. [4] § Materials and methods › Static images acquisition and analysis › Automated multi-worm morphological feature extraction from images. ↔ multiworm_feature_extraction/2_extract_wormcutouts.py, lines 35 to 45 · score 0.79 · pred_iou_thresh, stability_score_thresh, automatic mask, mask generation, CUDA, predictions · Report a mismatch
  5. [5] § Materials and methods › Calcium imaging in semi-restricted worms › Automated segmentation of axonal compartments and head bending extraction. ↔ singleworm_tracking/2_autoprompted_segmentation.py, lines 1 to 32 · score 0.75 · Hiera Large, prompt pool, video predictor, Full frame, prompt frames, checkpoint · Report a mismatch
  6. [6] § Materials and methods › Crawling recording and analysis › Automated single worm high-definition tracking. ↔ singleworm_tracking/4_path_analysis.py, lines 418 to 498 · score 0.73 · velocity vectors, Worm movement, movement classification, bouts, orientation, stationary · Report a mismatch
  7. [7] § Materials and methods › Droplet swimming recording and analysis › Automated quantitative analysis of swimming behavior. ↔ singleworm_tracking/2_autoprompted_segmentation.py, lines 1 to 32 · score 0.71 · sam2_hiera_large.pt, generic prompt frame, high definition, checkpoint, CUDA, cropped · Report a mismatch
  8. [8] § Materials and methods › Static images acquisition and analysis › Runtime benchmarking. ↔ multiworm_feature_extraction/2_extract_wormcutouts.py, lines 321 to 389 · score 0.70 · medial axis, connected component, pruned, perimeter, contour, width · Report a mismatch
  9. [9] § Materials and methods › Crawling recording and analysis › Automated single worm high-definition tracking. ↔ singleworm_tracking/3_shape_analysis.py, lines 529 to 659 · score 0.69 · recent history, Error correction, cumulative, sudden, Endpoint, tail · Report a mismatch
  10. [10] § Materials and methods › Calcium imaging in semi-restricted worms › Automated segmentation of axonal compartments and head bending extraction. ↔ RIA_calcium_imaging/2_crop_RIAregion.py, lines 1 to 24 · score 0.68 · RIA region, Hiera Large, video predictor, checkpoint, crop, SAM2 · Report a mismatch
  11. [11] § Materials and methods › Static images acquisition and analysis › Automated multi-worm morphological feature extraction from images. ↔ multiworm_feature_extraction/2_extract_wormcutouts.py, lines 321 to 389 · score 0.66 · medial axis, worm mask, longest, perimeter, graph, contour · Report a mismatch
  12. [12] § Materials and methods › Droplet swimming recording and analysis › Automated quantitative analysis of swimming behavior. ↔ droplet_swimming/2_fframe_segmentation.py, lines 1 to 27 · score 0.62 · sam2_hiera_large.pt, generic prompt frame, checkpoint, CUDA, Model, swimming · Report a mismatch
  13. [13] § Materials and methods › Static images acquisition and analysis › Automated multi-worm morphological feature extraction from images. ↔ multiworm_feature_extraction/2_extract_wormcutouts.py, lines 233 to 311 · score 0.59 · overlapping worm masks, Connected components, largest, classified · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 501 lines · 20 KB · MIT · 4 matches

  1. """
  2. This script uses the SAM model to segment the entire image.
  3. The resulting cutouts are classified as either 'worm_any' or 'not worm' by a fine-tuned classifier. "Worm_any" also includes partial worms.
  4. Metrics are extracted from the final worms and saved to a CSV file.
  5. """
  7. import torch
  8. import torch.nn as nn
  9. import torchvision
  10. from torchvision import transforms
  11. import sys
  12. import numpy as np
  13. import matplotlib.pyplot as plt
  14. import cv2
  15. from PIL import Image
  16. import skimage
  17. from skimage.measure import label
  18. from scipy.ndimage import convolve
  19. import glob
  20. import os
  21. import pickle
  22. from skimage.measure import label
  23. from scipy.ndimage import convolve
  24. import shutil
  25. sys.path.append("PATH_TO_CLONED_SAM2_REPO/segment-anything-2")
  26. from sam2.build_sam import build_sam2
  27. from sam2.automatic_mask_generator import SAM2AutomaticMaskGenerator
  28. torch.autocast(device_type="cuda", dtype=torch.bfloat16).__enter__()
  30. if torch.cuda.get_device_properties(0).major >= 8:
  31. # turn on tfloat32 for Ampere GPUs
  32. torch.backends.cuda.matmul.allow_tf32 = True
  33. torch.backends.cudnn.allow_tf32 = True
  35. #Setup the SAM model
  36. checkpoint = "./segment-anything-2/checkpoints/sam2_hiera_large.pt" #Checkpoint for the SAM model
  37. model_cfg = "sam2_hiera_l.yaml" #Configuration file for the SAM model
  38. sam2 = build_sam2(model_cfg, checkpoint, device ='cuda', apply_postprocessing=False)
  39. mask_generator = SAM2AutomaticMaskGenerator(sam2)
  40. mask_generator_2 = SAM2AutomaticMaskGenerator(
  41. model=sam2,
  42. pred_iou_thresh=0.85,
  43. stability_score_thresh=0.85,
  44. stability_score_offset=0.85
  45. )
  47. #Setup the worm classifier
  48. classifdevice = torch.device("cuda:0")
  49. classif_weights = torchvision.models.ViT_H_14_Weights.IMAGENET1K_SWAG_E2E_V1
  50. worm_noworm_classif_model = torchvision.models.vit_h_14(weights=classif_weights)
  51. num_ftrs = worm_noworm_classif_model.heads.head.in_features
  52. worm_noworm_classif_model.heads.head = nn.Linear(num_ftrs, 2)
  53. worm_noworm_classif_model = worm_noworm_classif_model.to(classifdevice)
  54. worm_noworm_classif_model.load_state_dict(torch.load('PATH_TO_WORM_NOWORM_CLASSIFIER_WITH_PERFECT_WEIGHTS.pth', map_location=classifdevice)) #https://huggingface.co/lillyguisnet/celegans-classifier-vit-h-14-finetuned
  56. worm_noworm_classif_model.eval()
  57. class_names = ["notworm", "worm_any"]
  58. data_transforms = {
  59. 'val': transforms.Compose([
  60. transforms.Resize(518),
  61. transforms.CenterCrop(518),
  62. transforms.ToTensor(),
  63. transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
  64. ]),
  65. }
  67. def show_anns(anns, borders=True):
  68. if len(anns) == 0:
  69. return
  70. sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True)
  71. ax = plt.gca()
  72. ax.set_autoscale_on(False)
  74. img = np.ones((sorted_anns[0]['segmentation'].shape[0], sorted_anns[0]['segmentation'].shape[1], 4))
  75. img[:,:,3] = 0
  76. for ann in sorted_anns:
  77. m = ann['segmentation']
  78. color_mask = np.concatenate([np.random.random(3), [0.5]])
  79. img[m] = color_mask
  80. if borders:
  81. contours, _ = cv2.findContours(m.astype(np.uint8),cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
  82. # Try to smooth contours
  83. contours = [cv2.approxPolyDP(contour, epsilon=0.01, closed=True) for contour in contours]
  84. cv2.drawContours(img, contours, -1, (0,0,1,0.4), thickness=1)
  86. ax.imshow(img)
  88. def is_on_edge(x, y, w, h, img_width, img_height):
  89. # Check left edge
  90. if x <= 0:
  91. return True
  92. # Check top edge
  93. if y <= 0:
  94. return True
  95. # Check right edge
  96. if (x + w) >= img_width - 1:
  97. return True
  98. # Check bottom edge
  99. if (y + h) >= img_height - 1:
  100. return True
  101. return False
  103. def get_valid_imaging_area(image, margin=5, max_iterations=100):
  104. """
  105. Find the actual microscope field of view in the image.
  106. """
  107. # Convert to grayscale if not already
  108. if len(image.shape) == 3:
  109. gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
  110. else:
  111. gray = image
  113. _, binary = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
  115. # Find contours
  116. contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
  117. if not contours:
  118. print("Warning: No valid imaging area found")
  119. return np.ones_like(gray, dtype=bool), False
  121. # Find the largest contour that's not the entire image
  122. valid_contours = [cnt for cnt in contours
  123. if 0.1 < cv2.contourArea(cnt) / (gray.shape[0] * gray.shape[1]) < 0.99]
  125. if not valid_contours:
  126. print("Warning: No valid contours found within acceptable size range")
  127. return np.ones_like(gray, dtype=bool), False
  129. largest_contour = max(valid_contours, key=cv2.contourArea)
  131. # Create mask of valid area
  132. valid_area_mask = np.zeros_like(gray, dtype=np.uint8)
  133. cv2.drawContours(valid_area_mask, [largest_contour], -1, 255, -1)
  135. # Erode the mask by margin pixels with iteration limit
  136. if margin > 0:
  137. kernel = np.ones((3, 3), np.uint8) # Using smaller kernel for more controlled erosion
  138. eroded_mask = valid_area_mask.copy()
  139. for _ in range(min(margin, max_iterations)):
  140. temp_mask = cv2.erode(eroded_mask, kernel)
  141. if np.sum(temp_mask) < 1000:
  142. break
  143. eroded_mask = temp_mask
  144. valid_area_mask = eroded_mask
  146. return valid_area_mask > 0, True
  148. def get_nonedge_masks(img_path):
  149. image = cv2.imread(img_path)
  150. image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
  152. img_height, img_width = image.shape[:2]
  154. # Generate masks
  155. masks2 = mask_generator_2.generate(image)
  157. valid_area, success = get_valid_imaging_area(image)
  159. nonedge_masks = []
  161. if success:
  162. # Use valid area method
  163. for mask in masks2:
  164. segmentation = mask['segmentation']
  165. if np.all(segmentation * valid_area == segmentation):
  166. nonedge_masks.append(segmentation)
  167. else:
  168. # Fall back to simple edge detection
  169. print(f"Falling back to simple edge detection for {img_path}")
  170. for mask in masks2:
  171. segmentation = mask['segmentation']
  172. coords = np.where(segmentation)
  173. y1, x1 = np.min(coords[0]), np.min(coords[1])
  174. y2, x2 = np.max(coords[0]), np.max(coords[1])
  175. h, w = (y2 - y1 + 1), (x2 - x1 + 1)
  177. if not is_on_edge(x1, y1, w, h, img_width, img_height):
  178. nonedge_masks.append(segmentation)
  180. return image, img_height, img_width, nonedge_masks
  183. def save_mask_cutouts(image, nonedge_masks, output_dir='PATH_TO_TEMP_CUTOUTS_DIR'):
  184. """
  185. Save cutouts of the masks from the image to the specified directory.
  186. Refreshes the output directory each time.
  187. """
  188. # Refresh temp directory
  189. if os.path.exists(output_dir):
  190. shutil.rmtree(output_dir)
  191. os.makedirs(output_dir, exist_ok=True)
  193. print(f"Saving {len(nonedge_masks)} non-edge cutouts to")
  195. for i, mask in enumerate(nonedge_masks):
  196. # Get bounding box coordinates of the mask
  197. coords = np.where(mask)
  198. y1, x1 = np.min(coords[0]), np.min(coords[1])
  199. y2, x2 = np.max(coords[0]), np.max(coords[1])
  201. # Create a 3D mask by repeating the 2D mask for each color channel
  202. mask_3d = np.repeat(mask[:, :, np.newaxis], 3, axis=2)
  204. # Apply mask to original image
  205. cutout = image * mask_3d
  207. # Crop to bounding box
  208. cutout = cutout[y1:y2+1, x1:x2+1]
  210. # Save the cutout as jpg
  211. cutout_path = os.path.join(output_dir, f'{i}.jpg')
  212. cv2.imwrite(cutout_path, cv2.cvtColor(cutout, cv2.COLOR_RGB2BGR))
  214. def classify_cutouts(nonedge_masks, cutouts_dir='PATH_TO_TEMP_CUTOUTS_DIR'):
  215. """
  216. Classify each cutout image as either 'worm' or 'not worm' using the pre-trained classifier.
  217. """
  218. classifications = []
  219. for i in range(len(nonedge_masks)):
  220. cutout_path = os.path.join(cutouts_dir, f'{i}.jpg')
  221. imgg = Image.open(cutout_path)
  222. imgg = data_transforms['val'](imgg)
  223. imgg = imgg.unsqueeze(0)
  224. imgg = imgg.to(classifdevice)
  226. outputs = worm_noworm_classif_model(imgg)
  227. _, preds = torch.max(outputs, 1)
  228. classifications.append(class_names[preds])
  230. return classifications
  233. def merge_and_clean_worm_masks(classifications, nonedge_masks, overlap_threshold=0.95, min_area=25):
  234. """
  235. Merge overlapping worm masks and clean the results by removing small regions and keeping only the largest connected component.
  236. Also checks for and removes masks with holes.
  237. """
  238. worm_masks = []
  239. for i, classification in enumerate(classifications):
  240. if classification == "worm_any":
  241. worm_masks.append(nonedge_masks[i])
  243. if worm_masks:
  244. # Initialize list to track which masks have been merged
  245. merged_masks = []
  246. final_masks = []
  248. # Compare each mask with every other mask
  249. for i in range(len(worm_masks)):
  250. if i in merged_masks:
  251. continue
  253. current_mask = worm_masks[i]
  254. current_area = np.sum(current_mask)
  255. merged = False
  257. for j in range(i + 1, len(worm_masks)):
  258. if j in merged_masks:
  259. continue
  261. other_mask = worm_masks[j]
  262. # Calculate overlap
  263. overlap = np.sum(current_mask & other_mask)
  264. overlap_ratio = overlap / min(current_area, np.sum(other_mask))
  266. # If overlap is more than threshold, merge the masks
  267. if overlap_ratio > overlap_threshold:
  268. current_mask = current_mask | other_mask
  269. current_area = np.sum(current_mask)
  270. merged_masks.append(j)
  271. merged = True
  273. final_masks.append(current_mask)
  275. # Clean final masks - remove regions smaller than min_area pixels and handle discontinuous segments
  276. worm_masks = []
  277. for i, mask in enumerate(final_masks):
  278. if np.sum(mask) >= min_area:
  279. # Check for holes using contour hierarchy
  280. contours, hierarchy = cv2.findContours((mask * 255).astype(np.uint8),
  281. cv2.RETR_TREE,
  282. cv2.CHAIN_APPROX_SIMPLE)
  284. has_holes = False
  285. if hierarchy is not None:
  286. hierarchy = hierarchy[0] # Get the first dimension
  287. for h in hierarchy:
  288. if h[3] >= 0: # If has parent, it's a hole
  289. has_holes = True
  290. print(f"Skipping mask {i} due to holes in the mask")
  291. break
  293. if not has_holes:
  294. # Find connected components in the mask
  295. num_labels, labels = cv2.connectedComponents(mask.astype(np.uint8))
  297. if num_labels > 2: # More than one segment (label 0 is background)
  298. # Get sizes of each segment
  299. unique_labels, label_counts = np.unique(labels[labels != 0], return_counts=True)
  300. # Keep only the largest segment
  301. largest_label = unique_labels[np.argmax(label_counts)]
  302. mask = (labels == largest_label).astype(np.uint8)
  304. worm_masks.append(mask)
  306. num_distinct_worms = len(worm_masks)
  307. print(f"Number of distinct worm regions: {num_distinct_worms}")
  308. else:
  309. num_distinct_worms = 0
  311. return worm_masks, num_distinct_worms
  314. def filter_worms(allworms_metrics, threshold):
  315. filtered_metrics = []
  316. for worm in allworms_metrics:
  317. if worm['area'] > threshold * np.mean([worm['area'] for worm in allworms_metrics]):
  318. filtered_metrics.append(worm)
  319. return filtered_metrics
  321. def extract_worm_metrics(worm_masks, img_path, img_height, img_width, threshold=0.75):
  322. """
  323. Extract metrics for each worm mask including area, perimeter, medial axis measurements, etc.
  324. """
  325. # Get image ID (filename without extension)
  326. img_id = os.path.splitext(os.path.basename(img_path))[0]
  328. allworms_metrics = []
  329. for i, npmask in enumerate(worm_masks):
  330. print(f"Processing worm {i}")
  332. num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats((npmask * 255).astype(np.uint8), connectivity=8)
  333. largest_label = 1 + np.argmax(stats[1:, cv2.CC_STAT_AREA])
  334. largest_component_mask = (labels == largest_label).astype(np.uint8)
  336. area = np.sum(largest_component_mask)
  338. contours, hierarchy = cv2.findContours((largest_component_mask*255).astype(np.uint8), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
  340. perimeter = cv2.arcLength(contours[0], True)
  342. # Get medial axis and distance transform
  343. medial_axis, distance = skimage.morphology.medial_axis(largest_component_mask > 0, return_distance=True)
  344. structuring_element = np.array([[1, 1, 1], [1, 10, 1], [1, 1, 1]], dtype=np.uint8)
  345. neighbours = convolve(medial_axis.astype(np.uint8), structuring_element, mode='constant', cval=0)
  346. end_points = np.where(neighbours == 11, 1, 0)
  347. branch_points = np.where(neighbours > 12, 1, 0)
  348. labeled_branches = label(branch_points, connectivity=2)
  349. branch_indices = np.argwhere(labeled_branches > 0)
  350. end_indices = np.argwhere(end_points > 0)
  351. indices = np.concatenate((branch_indices, end_indices), axis=0)
  353. # Find longest path through medial axis
  354. paths = []
  355. for start in range(len(indices)):
  356. for end in range(len(indices)):
  357. startid = tuple(indices[start])
  358. endid = tuple(indices[end])
  359. route, weight = skimage.graph.route_through_array(np.invert(medial_axis), startid, endid)
  360. length = len(route)
  361. paths.append([startid, endid, length, route, weight])
  363. longest_length = max(paths, key=lambda x: x[2])
  364. pruned_mediala = np.zeros((img_height, img_width), dtype=np.uint8)
  365. for coord in range(len(longest_length[3])):
  366. pruned_mediala[longest_length[3][coord]] = 1
  368. # Get measurements along medial axis
  369. medial_axis_distances_sorted = [distance[pt[0], pt[1]] for pt in longest_length[3]]
  370. medialaxis_length_list = 0 + np.arange(0, len(medial_axis_distances_sorted))
  371. pruned_medialaxis_length = np.sum(pruned_mediala)
  372. mean_wormwidth = np.mean(medial_axis_distances_sorted)
  373. mid_length = medial_axis_distances_sorted[int(len(medial_axis_distances_sorted)/2)]
  375. worm_metrics = {
  376. "img_id": img_id,
  377. "worm_id": i,
  378. "area": area,
  379. "perimeter": perimeter,
  380. "medial_axis_distances_sorted": medial_axis_distances_sorted,
  381. "medialaxis_length_list": np.ndarray.tolist(medialaxis_length_list),
  382. "pruned_medialaxis_length": pruned_medialaxis_length,
  383. "mean_wormwidth": mean_wormwidth,
  384. "mid_length_width": mid_length,
  385. "mask": largest_component_mask
  386. }
  387. allworms_metrics.append(worm_metrics)
  389. return filter_worms(allworms_metrics, threshold = threshold)
  392. def save_worms(allworms_metrics, original_image=None, cutouts_dir='PATH_TO_FINAL_CUTOUTS_DIR',
  393. metrics_dir='PATH_TO_FINAL_METRICS_DIR'):
  394. """
  395. Filter worms by area and save the filtered cutouts and metrics.
  396. """
  397. if not allworms_metrics:
  398. print("No worm metrics provided")
  399. return []
  401. img_id = allworms_metrics[0]["img_id"]
  403. # Save cutouts of filtered worms for visualization
  404. for i, worm in enumerate(allworms_metrics):
  405. cutout_path = os.path.join(cutouts_dir, f'{img_id}_worm_{i}.png')
  406. cutout_name = f'{img_id}_worm_{i}' # The name that will appear on the image
  408. if original_image is not None:
  409. overlay = original_image.copy()
  410. overlay[worm["mask"] > 0] = [0, 255, 0] # Green color
  412. alpha = 0.4 # Back to original 40% transparency
  413. blended = cv2.addWeighted(original_image, 1 - alpha, overlay, alpha, 0)
  415. font = cv2.FONT_HERSHEY_SIMPLEX
  416. cv2.putText(blended, cutout_name, (10, 30), font, 1, (255, 255, 255), 2)
  418. cv2.imwrite(cutout_path, cv2.cvtColor(blended, cv2.COLOR_RGB2BGR))
  419. else:
  420. # Fall back to saving just the mask if original image not provided
  421. cv2.imwrite(cutout_path, (worm["mask"] * 255).astype(np.uint8))
  423. # Save metrics as pickle using img_id as filename
  424. metrics_path = os.path.join(metrics_dir, f'{img_id}.pkl')
  425. with open(metrics_path, 'wb') as f:
  426. pickle.dump(allworms_metrics, f)
  427. print(f"Saved filtered metrics to {metrics_path}")
  429. return allworms_metrics
  433. def process_folder(input_folder, temp_cutouts_dir='PATH_TO_TEMP_CUTOUTS_DIR',
  434. final_cutouts_dir='PATH_TO_FINAL_CUTOUTS_DIR',
  435. metrics_dir='PATH_TO_FINAL_METRICS_DIR',
  436. noworms_file='PATH_TO_NOWORMS_FILE.csv'):
  437. """
  438. Process all images in a folder through the complete worm analysis pipeline.
  439. """
  441. # Create/check noworms CSV file
  442. if not os.path.exists(noworms_file):
  443. with open(noworms_file, 'w') as f:
  444. f.write('image_path\n')
  446. # Create/check final cutouts directory
  447. if not os.path.exists(final_cutouts_dir):
  448. os.makedirs(final_cutouts_dir)
  450. # Create/check metrics directory
  451. if not os.path.exists(metrics_dir):
  452. os.makedirs(metrics_dir)
  454. # Get list of image files
  455. image_files = glob.glob(os.path.join(input_folder, '*.jpg'))
  457. # Process each image
  458. for img_path in image_files:
  459. print(f"\nProcessing {img_path}")
  460. try:
  461. # Extract masks
  462. image, img_height, img_width, nonedge_masks = get_nonedge_masks(img_path)
  463. if len(nonedge_masks) == 0:
  464. print(f"No valid masks found in {img_path}")
  465. print(f"Number of worms in image: 0")
  466. with open(noworms_file, 'a') as f:
  467. f.write(f'{img_path}\n')
  468. continue
  470. save_mask_cutouts(image, nonedge_masks, temp_cutouts_dir)
  472. classifications = classify_cutouts(nonedge_masks, temp_cutouts_dir)
  474. worm_masks, num_distinct_worms = merge_and_clean_worm_masks(
  475. classifications, nonedge_masks, temp_cutouts_dir)
  476. if num_distinct_worms == 0:
  477. print(f"No worms detected in {img_path}")
  478. print(f"Number of worms in image: 0")
  479. with open(noworms_file, 'a') as f:
  480. f.write(f'{img_path}\n')
  481. continue
  483. worm_metrics = extract_worm_metrics(worm_masks, img_path, img_height, img_width)
  485. print(f"Final number of worms in image: {len(worm_metrics)}")
  487. save_worms(worm_metrics, original_image=image, cutouts_dir=final_cutouts_dir, metrics_dir=metrics_dir)
  489. except Exception as e:
  490. print(f"Error processing {img_path}: {str(e)}")
  491. print(f"Number of worms in image: 0")
  492. continue
  494. print(f"\nAnalysis complete. Processed {len(image_files)} images.")
  495. print(f"Results saved to {metrics_dir}")
  499. input_folder = 'PATH_TO_INPUT_FOLDER'
  501. process_folder(input_folder)

2_extract_wormcutouts.py at commit 92d4d38, under MIT · at the source

Overview

Authors: Aurélie Guisnet1, Michael Hendricks1
ORCID iDs: Aurélie Guisnet, Michael Hendricks
  1. Department of Biology, McGill University, Montreal, Quebec, Canada
Institutions: McGill University (Canada)
Journal: PLoS computational biology, volume 22, issue 7, article e1013441
Dates: received 15 August 2025; accepted 26 June 2026; published online 22 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1371/journal.pcbi.1013441 · PMID 42485426 · PMCID PMC13492816 · OpenAlex W7170066936
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: structural MRI / diffusion (modality), optical imaging (calcium, voltage, 2-photon) (modality), C. elegans (organism), methods / tools (subfield)
Methods: Spectral & time-frequency, Connectivity, Graphs, fMRI & imaging, Single-unit activity, calcium imaging, Smoothing, state filtering, decompositions, Machine learning
MeSH: Caenorhabditis elegans*, Image Processing, Computer-Assisted*, Algorithms, Animals, Computational Biology, Neurons, Software (* major topic)
Journal subjects: Biology and Life Sciences, Physiology, Biological Locomotion, Swimming, Research and Analysis Methods, Imaging Techniques, Neuroimaging, Calcium Imaging, Neuroscience, Animal Studies, Experimental Organism Systems, Model Organisms, Caenorhabditis Elegans, Animal Models, Organisms, Eukaryota, Animals, Invertebrates, Nematoda, Caenorhabditis, Zoology, Fluorescence Imaging, Crawling, Developmental Biology, Morphogenesis, Morphogenic Segmentation, Psychology, Behavior, Animal Behavior, Social Sciences, Image Analysis
Topic: Genetics, Aging, and Longevity in Model Organisms (Aging, Biochemistry, Genetics and Molecular Biology), according to OpenAlex
Funding: National Science and Engineering Research Council (RGPIN/05117-2014); Gouvernement du Canada | Canadian Institutes of Health Research (IRSC) (PJT-155980); Canadian Foundation for Innovation (32581); Canada Research Chairs program (950-231541); Fonds de recherche du Québec – Nature et technologies (300853)
Citations: not cited yet (Europe PMC); 66 references in the paper

Abstract

Quantitative phenotyping of Caenorhabditis elegans is essential across numerous fields, yet data extraction remains a significant analytical bottleneck. Traditional segmentation methods based on pixel-intensity thresholding are highly sensitive to variations in imaging conditions and often fail in the presence of noise, overlaps, or uneven illumination. These failures necessitate meticulous experimental setups, expensive hardware, or extensive manual curation, which reduces throughput and introduces bias. Here, we introduce TWARDIS (Tools for Worm Automated Recognition & Dynamic Imaging System), a modular, Python-based analysis suite that leverages large foundation vision models, specifically the Segment Anything Models (SAM and SAM2) and a fine-tuned vision transformer classifier, to overcome some of these limitations. We demonstrate the versatility of an AI compound system approach across diverse modalities. For static morphological analysis, TWARDIS successfully resolved overlapping worms in noisy images without human intervention, showing a 0.999 correlation with manual segmentation. In behavioral assays (swimming and crawling), the pipeline enabled high-definition postural analysis even in low-resolution, wide-field recordings where the worm occupied only ~0.25% of the field of view, accurately resolving complex postures without frame rejection. Finally, when applied to calcium imaging of semi-restricted animals, TWARDIS provided precise, frame-by-frame segmentation of neural compartments, reducing the artificial signal flattening common in traditional region-of-interest-based approaches and enabling the extraction of biologically accurate, absolute head positions. The system’s hardware-scalable architecture and modular design ensure both current accessibility and future improvements without restructuring. By automating some of the most time-consuming aspects of image analysis, TWARDIS removes many critical bottlenecks and tradeoffs in C. elegans research, enabling researchers to focus on biological questions rather than technical image processing challenges.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 13 matches between paragraphs and lines of code.

lillyguisnet/TWARDISv0.1

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 92d4d382bf0e41b408cbc3389ea22f196537a56f, 17 May 2026
Languages: Python (17)
Size: 26 files, 17 scripts
Software Heritage: not archived
Found in: the text, “Hardware and code”
Holds: README, license file
Not found: CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (15 files), OpenCV (12 files), Matplotlib (10 files), Pillow (9 files), PyTorch (8 files), SciPy (8 files), h5py (6 files), scikit-image (4 files), NetworkX (2 files), pandas (2 files), tifffile (2 files)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
19 files
At the source: github.com/lillyguisnet/TWARDISv0.1

huggingface.co/lillyguisnet/celegans-classifier-vit-h-14-finetuned

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 111e51be4ea7bd042252a11ab632839cb156b001, 24 March 2025
Size: 4 files, 0 scripts
Software Heritage: not archived
Found in: the text, “Hardware and code”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
At the source: huggingface.co/lillyguisnet/celegans-classifier-vit-h-14-finetuned

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 17 scripts, each with its path and the digest of its content;
  • 13 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data Availability

All relevant data and code are within the paper, its Supporting Information files, and on GitHub at https://github.com/lillyguisnet/TWARDISv0.1. The fine-tuned worm classifier model is available on HuggingFace at https://huggingface.co/lillyguisnet/celegans-classifier-vit-h-14-finetuned.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 7 MeSH terms, 5 funders, 46 references.

Cite

This paper

Guisnet, A., & Hendricks, M. (2026). Large vision model framework for automated C. elegans analysis: From static morphometry to dynamic neural activity. PLoS computational biology, 22(7), e1013441. https://doi.org/10.1371/journal.pcbi.1013441

BibTeX

@article{guisnet2026large,
author = {Guisnet, Aurélie and Hendricks, Michael},
title = {{Large vision model framework for automated C. elegans analysis: From static morphometry to dynamic neural activity}},
journal = {PLoS computational biology},
year = {2026},
month = jul,
volume = {22},
number = {7},
pages = {e1013441},
publisher = {PLOS},
issn = {1553-734X},
doi = {10.1371/journal.pcbi.1013441},
url = {https://doi.org/10.1371/journal.pcbi.1013441},
pmid = {42485426},
pmcid = {PMC13492816}
}

RIS

TY - JOUR
AU - Guisnet, Aurélie
AU - Hendricks, Michael
TI - Large vision model framework for automated C. elegans analysis: From static morphometry to dynamic neural activity
T2 - PLoS computational biology
J2 - PLoS Comput Biol
PY - 2026
DA - 2026/07/22
VL - 22
IS - 7
SP - e1013441
SN - 1553-734X
PB - PLOS
DO - 10.1371/journal.pcbi.1013441
UR - https://doi.org/10.1371/journal.pcbi.1013441
LA - en
ER -

CSL-JSON

{
"id": "10.1371/journal.pcbi.1013441",
"type": "article-journal",
"title": "Large vision model framework for automated C. elegans analysis: From static morphometry to dynamic neural activity",
"container-title": "PLoS computational biology",
"author": [
{
"family": "Guisnet",
"given": "Aurélie"
},
{
"family": "Hendricks",
"given": "Michael"
}
],
"container-title-short": "PLoS Comput Biol",
"volume": "22",
"issue": "7",
"page": "e1013441",
"DOI": "10.1371/journal.pcbi.1013441",
"PMID": "42485426",
"PMCID": "PMC13492816",
"ISSN": "1553-734X",
"publisher": "PLOS",
"URL": "https://doi.org/10.1371/journal.pcbi.1013441",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
22
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41467-026-72709-w [code]
An epifluorescence microscope design for naturalistic behavior and cellular activity in freely moving Caenorhabditis elegans.
Journal: Nature communications
In common: tifffile, OpenCV, scikit-image, 7 other tools, C. elegans, 3 references, 2 authors
[2] doi:10.1371/journal.pone.0331188 [code]
VNC-Dist: A machine learning-based semi-automated pipeline for quantification of neuronal position in the C. elegans ventral nerve cord.
Journal: PloS one
In common: OpenCV, scikit-image, Pillow, 5 other tools, C. elegans, methods / tools, 6 references
[3] doi:10.1038/s41467-026-72710-3 [code]
A modular multi-color fluorescence microscope for simultaneous tracking of cellular activity and behavior.
Journal: Nature communications
In common: tifffile, OpenCV, scikit-image, 5 other tools, C. elegans, optical imaging (calcium, voltage, 2-photon), 3 references
[4] doi:10.1186/s12859-025-06150-z [code]
CRISP: correlation-refined image segmentation process.
Journal: BMC bioinformatics
In common: tifffile, NetworkX, scikit-image, 7 other tools, optical imaging (calcium, voltage, 2-photon), methods / tools, 1 reference
[5] doi:10.7554/elife.80918 [code]
MorphoFeatures for unsupervised exploration of cell types, tissues, and organs in volume electron microscopy.
Journal: eLife
In common: tifffile, NetworkX, scikit-image, 6 other tools, 4 references
[6] doi:10.1038/s41467-023-44363-z [code]
Multimodal measures of spontaneous brain activity reveal both common and divergent patterns of cortical functional organization.
Journal: Nature communications
In common: NetworkX, scikit-image, h5py, 5 other tools, optical imaging (calcium, voltage, 2-photon), 4 references
[7] doi:10.1364/boe.600665 [code]
NeuroSeg-MF: robust neuron segmentation in two-photon Ca&lt;sup&gt;2+&lt;/sup&gt; imaging using multi-feature fusion and detection-guided SAM.
Journal: Biomedical optics express
In common: tifffile, OpenCV, Pillow, 5 other tools, optical imaging (calcium, voltage, 2-photon), methods / tools, 2 references
[8] doi:10.1016/j.neuroimage.2023.120109 [code]
Benchmarking explanation methods for mental state decoding with deep learning models.
Journal: NeuroImage
In common: NetworkX, scikit-image, h5py, 6 other tools, methods / tools, 3 references
[9] doi:10.1364/boe.605322 [code]
Generalized plaque digitization framework for multi-dimensional mesoscopic images.
Journal: Biomedical optics express
In common: tifffile, OpenCV, scikit-image, 7 other tools, 1 reference
[10] doi:10.1016/j.patter.2026.101538 [code]
A multi-modal foundation model for brain disease diagnosis and medical imaging.
Journal: Patterns (New York, N.Y.)
In common: tifffile, OpenCV, scikit-image, 7 other tools, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441.svg)](https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441.svg
   :target: https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441"><img src="https://openscicode.org/badge/doi/10.1371/journal.pcbi.1013441.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.