OSCR

Brain tumor detection on magnetic resonance imaging scans using the artificial intelligence-based You Only Look Once algorithm.

Code ↔ Paper

11 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 11 matches · 2 of them tie a paragraph to a whole file without given lines: weak matches, whose lines are not tinted
  1. [1] § Methods ↔ shenduxuexilvcai_02/ultralytics/nn/tasks.py, lines 1047 to 1114 · score 0.65 · A2C2f, C2TSSA DYT, suppressing, backbone, DFFN, thresholds · Report a mismatch
  2. [2] § Methods › C2TSSA-DYT module ↔ shenduxuexilvcai_02/ultralytics/models/sam/modules/encoders.py, lines 501 to 642 · score 0.64 · feature fusion, convolutional layer, channel dimension, fused, module, detection · Report a mismatch
  3. [3] § Methods › A2C2f-DFFN module ↔ shenduxuexilvcai_02/ultralytics/nn/extra_modules/block.py, lines 1 to 70 · score 0.59 · edge information, additive, deep, gating, DFFN, fusion · Report a mismatch
  4. [4] § Methods › C2TSSA-DYT module ↔ shenduxuexilvcai_02/ultralytics/nn/tasks.py, lines 1116 to 1175 · score 0.58 · C2TSSA DYT, A2C2f, DFFN, dynamic, global, module · Report a mismatch
  5. [5] § Experimental analyses › Visualization of results ↔ shenduxuexilvcai_02/ultralytics/models/yolo/classify/val.py, the whole file · a weak match · score 0.58 · confusion matrices, yolov8n, classification, metrics, accuracy, detections · Report a mismatch
  6. [6] § Experimental analyses › Algorithm comparison results ↔ shenduxuexilvcai_02/ultralytics/models/rtdetr/train.py, lines 14 to 102 · score 0.54 · inference speed, RT DETR, YOLO, detections · Report a mismatch
  7. [7] § Experimental analyses › Visualization of results ↔ shenduxuexilvcai_02/ultralytics/models/rtdetr/predict.py, lines 11 to 84 · score 0.52 · bounding boxes, RT DETR, confidence, scoring, accuracy, detection · Report a mismatch
  8. [8] § Experiment › Dataset › Experimental environment and hyperparameter configuration ↔ shenduxuexilvcai_02/ultralytics/engine/tuner.py, lines 70 to 112 · score 0.51 · weight decay, momentum, configuration, hyperparameter · Report a mismatch
  9. [9] § Methods ↔ shenduxuexilvcai_02/ultralytics/nn/tasks.py, lines 1047 to 1114 · score 0.51 · A2C2f, C2TSSA DYT, backbone, DFFN, modules · Report a mismatch
  10. [10] § Experiment › Dataset › Experimental environment and hyperparameter configuration ↔ shenduxuexilvcai_02/ultralytics/utils/tuner.py, the whole file · a weak match · score 0.50 · weight decay, momentum, hyperparameter, GPU, trained, models · Report a mismatch
  11. [11] § Methods › A2C2f-DFFN module ↔ shenduxuexilvcai_02/ultralytics/models/sam/modules/blocks.py, lines 877 to 974 · score 0.50 · residual connection, propagated, network, layer, global, dimensional · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 1,506 lines · 67 KB · no license · 3 matches

  1. # Ultralytics YOLO 🚀, AGPL-3.0 license
  3. import contextlib
  4. import pickle
  5. import re
  6. import types
  7. from copy import deepcopy
  8. from pathlib import Path
  10. import timm
  11. import torch
  12. import torch.nn as nn
  14. from ultralytics.nn.modules import (
  15. AIFI,
  16. C1,
  17. C2,
  18. C2PSA,
  19. C3,
  20. C3TR,
  21. ELAN1,
  22. OBB,
  23. PSA,
  24. SPP,
  25. SPPELAN,
  26. SPPF,
  27. AConv,
  28. # ADown,
  29. Bottleneck,
  30. BottleneckCSP,
  31. C2f,
  32. C2fAttn,
  33. C2fCIB,
  34. C2fPSA,
  35. C3Ghost,
  36. C3k2,
  37. C3x,
  38. # CBFuse,
  39. # CBLinear,
  40. Classify,
  41. Concat,
  42. Conv,
  43. Conv2,
  44. ConvTranspose,
  45. Detect,
  46. DWConv,
  47. DSConv,
  48. DWConvTranspose2d,
  49. Focus,
  50. GhostBottleneck,
  51. GhostConv,
  52. HGBlock,
  53. HGStem,
  54. ImagePoolingAttn,
  55. Pose,
  56. RepC3,
  57. RepConv,
  58. # RepNCSPELAN4,
  59. RepVGGDW,
  60. ResNetLayer,
  61. RTDETRDecoder,
  62. SCDown,
  63. Segment,
  64. WorldDetect,
  65. v10Detect,
  66. SpatialAttention,
  67. CBAM,
  68. A2C2f
  69. )
  70. from ultralytics.nn.extra_modules import *
  71. from ultralytics.utils import DEFAULT_CFG_DICT, DEFAULT_CFG_KEYS, LOGGER, colorstr, emojis, yaml_load
  72. from ultralytics.utils.checks import check_requirements, check_suffix, check_yaml
  73. from ultralytics.utils.loss import (
  74. E2EDetectLoss,
  75. v8ClassificationLoss,
  76. v8DetectionLoss,
  77. v8OBBLoss,
  78. v8PoseLoss,
  79. v8SegmentationLoss,
  80. E2EFGIDetectLoss,
  81. )
  82. from ultralytics.utils.ops import make_divisible
  83. from ultralytics.utils.plotting import feature_visualization
  84. from ultralytics.utils.torch_utils import (
  85. fuse_conv_and_bn,
  86. fuse_deconv_and_bn,
  87. initialize_weights,
  88. intersect_dicts,
  89. model_info,
  90. scale_img,
  91. time_sync,
  92. get_num_params
  93. )
  95. from ultralytics.nn.backbone.convnextv2 import *
  96. from ultralytics.nn.backbone.fasternet import *
  97. from ultralytics.nn.backbone.efficientViT import *
  98. from ultralytics.nn.backbone.EfficientFormerV2 import *
  99. from ultralytics.nn.backbone.VanillaNet import *
  100. from ultralytics.nn.backbone.revcol import *
  101. from ultralytics.nn.backbone.lsknet import *
  102. from ultralytics.nn.backbone.SwinTransformer import *
  103. from ultralytics.nn.backbone.repvit import *
  104. from ultralytics.nn.backbone.CSwomTramsformer import *
  105. from ultralytics.nn.backbone.UniRepLKNet import *
  106. from ultralytics.nn.backbone.TransNext import *
  107. from ultralytics.nn.backbone.rmt import *
  108. from ultralytics.nn.backbone.pkinet import *
  109. from ultralytics.nn.backbone.mobilenetv4 import *
  110. from ultralytics.nn.backbone.starnet import *
  111. from ultralytics.nn.backbone.inceptionnext import *
  112. from ultralytics.nn.extra_modules.mobileMamba.mobilemamba import *
  113. from ultralytics.nn.backbone.MambaOut import *
  115. try:
  116. import thop
  117. except ImportError:
  118. thop = None
  121. DETECT_CLASS = (Detect, Detect_DyHead, Detect_AFPN_P2345, Detect_AFPN_P2345_Custom, Detect_AFPN_P345, Detect_AFPN_P345_Custom, Detect_Efficient, DetectAux, Detect_SEAM, Detect_MultiSEAM,
  122. Detect_DyHeadWithDCNV3, Detect_DyHeadWithDCNV4, Detect_DyHead_Prune, Detect_LSCD, Detect_TADDH, Detect_LADH, Detect_LSCSBD, Detect_LSDECD, Detect_RSCD, Detect_FGIF)
  123. V10_DETECT_CLASS = (v10Detect,)
  124. SEGMENT_CLASS = (Segment, Segment_Efficient, Segment_LSCD, Segment_TADDH, Segment_LADH, Segment_LSCSBD, Segment_LSDECD, Segment_RSCD)
  125. POSE_CLASS = (Pose, Pose_LSCD, Pose_TADDH, Pose_LADH, Pose_LSCSBD, Pose_LSDECD, Pose_RSCD)
  126. OBB_CLASS = (OBB, OBB_LSCD, OBB_TADDH, OBB_LADH, OBB_LSCSBD, OBB_LSDECD, OBB_RSCD)
  127. C3K2_CLASS = (C3k2_Faster, C3k2_ODConv, C3k2_Faster_EMA, C3k2_DBB, C3k2_DeepDBB, C3k2_WDBB, C3k2_CloAtt, C3k2_SCConv, C3k2_ScConv, C3k2_EMSC, C3k2_EMSCP, C3k2_KW, C3k2_DySnakeConv, C3k2_DCNv2, C3k2_DCNv3,
  128. C3k2_OREPA, C3k2_REPVGGOREPA, C3k2_DCNv2_Dynamic, C3k2_ContextGuided, C3k2_MSBlock, C3k2_DLKA, C3k2_EMBC, C3k2_DAttention, C3k2_Parc, C3k2_DWR, C3k2_RFAConv, C3k2_RFCBAMConv, C3k2_RFCAConv,
  129. C3k2_FocusedLinearAttention, C3k2_MLCA, C3k2_AKConv, C3k2_UniRepLKNetBlock, C3k2_DRB, C3k2_DWR_DRB, C3k2_AggregatedAtt, C3k2_DCNv4, C3k2_SWC, C3k2_iRMB, C3k2_iRMB_Cascaded, C3k2_iRMB_DRB, C3k2_iRMB_SWC,
  130. C3k2_VSS, C3k2_LVMB, C3k2_DynamicConv, C3k2_GhostDynamicConv, C3k2_RVB, C3k2_RVB_SE, C3k2_RVB_EMA, C3k2_RetBlock, C3k2_PKIModule, C3k2_FADC, C3k2_PPA, C3k2_Faster_CGLU, C3k2_Star, C3k2_Star_CAA,
  131. C3k2_KAN, C3k2_EIEM, C3k2_DEConv, C3k2_SMPCGLU, C3k2_Heat, C3k2_WTConv, C3k2_FMB, C3k2_gConv, C3k2_AdditiveBlock, C3k2_AdditiveBlock_CGLU, C3k2_MSMHSA_CGLU, C3k2_MogaBlock, C3k2_SHSA, C3k2_SHSA_CGLU,
  132. C3k2_SMAFB, C3k2_SMAFB_CGLU, C3k2_IdentityFormer, C3k2_RandomMixing, C3k2_PoolingFormer, C3k2_ConvFormer, C3k2_CaFormer, C3k2_IdentityFormerCGLU, C3k2_RandomMixingCGLU, C3k2_PoolingFormerCGLU, C3k2_ConvFormerCGLU, C3k2_CaFormerCGLU,
  133. C3k2_FFCM, C3k2_MutilScaleEdgeInformationEnhance, C3k2_SFHF, C3k2_MSM, C3k2_MutilScaleEdgeInformationSelect, C3k2_HDRAB, C3k2_RAB, C3k2_LFE, C3k2_FCA_CTA, C3k2_FCA_SFA, C3k2_IDWC, C3k2_IDWB, C3k2_PConv, C3k2_EMA,
  134. C3k2_CAMixer, C3k2_HFERB, C3k2_DTAB, C3k2_JDPM, C3k2_ETB, C3k2_FDT, C3k2_AP, C3k2_ELGCA, C3k2_ELGCACGLU, C3k2_Strip, C3k2_StripCGLU, C3k2_Kat, C3k2_Faster_KAN, C3k2_DCMB, C3k2_DCMB_KAN, C3k2_GlobalFilter, C3k2_DynamicFilter,
  135. C3k2_TSSA, C3k2_SAVSS, C3k2_MobileMamba, C3k2_MambaOut
  136. )
  138. class BaseModel(nn.Module):
  139. """The BaseModel class serves as a base class for all the models in the Ultralytics YOLO family."""
  141. def forward(self, x, *args, **kwargs):
  142. """
  143. Perform forward pass of the model for either training or inference.
  145. If x is a dict, calculates and returns the loss for training. Otherwise, returns predictions for inference.
  147. Args:
  148. x (torch.Tensor | dict): Input tensor for inference, or dict with image tensor and labels for training.
  149. *args (Any): Variable length argument list.
  150. **kwargs (Any): Arbitrary keyword arguments.
  152. Returns:
  153. (torch.Tensor): Loss if x is a dict (training), or network predictions (inference).
  154. """
  155. if isinstance(x, dict): # for cases of training and validating while training.
  156. return self.loss(x, *args, **kwargs)
  157. return self.predict(x, *args, **kwargs)
  159. def predict(self, x, profile=False, visualize=False, augment=False, embed=None):
  160. """
  161. Perform a forward pass through the network.
  163. Args:
  164. x (torch.Tensor): The input tensor to the model.
  165. profile (bool): Print the computation time of each layer if True, defaults to False.
  166. visualize (bool): Save the feature maps of the model if True, defaults to False.
  167. augment (bool): Augment image during prediction, defaults to False.
  168. embed (list, optional): A list of feature vectors/embeddings to return.
  170. Returns:
  171. (torch.Tensor): The last output of the model.
  172. """
  173. if augment:
  174. return self._predict_augment(x)
  175. return self._predict_once(x, profile, visualize, embed)
  177. def _predict_once(self, x, profile=False, visualize=False, embed=None):
  178. """
  179. Perform a forward pass through the network.
  181. Args:
  182. x (torch.Tensor): The input tensor to the model.
  183. profile (bool): Print the computation time of each layer if True, defaults to False.
  184. visualize (bool): Save the feature maps of the model if True, defaults to False.
  185. embed (list, optional): A list of feature vectors/embeddings to return.
  187. Returns:
  188. (torch.Tensor): The last output of the model.
  189. """
  190. y, dt, embeddings = [], [], [] # outputs
  191. for idx, m in enumerate(self.model):
  192. if m.f != -1: # if not from previous layer
  193. x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f] # from earlier layers
  194. if profile:
  195. self._profile_one_layer(m, x, dt)
  196. if hasattr(m, 'backbone'):
  197. x = m(x)
  198. for _ in range(5 - len(x)):
  199. x.insert(0, None)
  200. for i_idx, i in enumerate(x):
  201. if i_idx in self.save:
  202. y.append(i)
  203. else:
  204. y.append(None)
  205. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{", ".join([str(x_.size()) for x_ in x if x_ is not None])}')
  206. x = x[-1]
  207. else:
  208. x = m(x) # run
  209. y.append(x if m.i in self.save else None) # save output
  211. # if type(x) in {list, tuple}:
  212. # if idx == (len(self.model) - 1):
  213. # if type(x[1]) is dict:
  214. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{", ".join([str(x_.size()) for x_ in x[1]["one2one"]])}')
  215. # else:
  216. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{", ".join([str(x_.size()) for x_ in x[1]])}')
  217. # else:
  218. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{", ".join([str(x_.size()) for x_ in x if x_ is not None])}')
  219. # elif type(x) is dict:
  220. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{", ".join([str(x_.size()) for x_ in x["one2one"]])}')
  221. # else:
  222. # if not hasattr(m, 'backbone'):
  223. # print(f'layer id:{idx:>2} {m.type:>50} output shape:{x.size()}')
  225. if visualize:
  226. feature_visualization(x, m.type, m.i, save_dir=visualize)
  227. if embed and m.i in embed:
  228. embeddings.append(nn.functional.adaptive_avg_pool2d(x, (1, 1)).squeeze(-1).squeeze(-1)) # flatten
  229. if m.i == max(embed):
  230. return torch.unbind(torch.cat(embeddings, 1), dim=0)
  231. return x
  233. def _predict_augment(self, x):
  234. """Perform augmentations on input image x and return augmented inference."""
  235. LOGGER.warning(
  236. f"WARNING ⚠️ {self.__class__.__name__} does not support 'augment=True' prediction. "
  237. f"Reverting to single-scale prediction."
  238. )
  239. return self._predict_once(x)
  241. def _profile_one_layer(self, m, x, dt):
  242. """
  243. Profile the computation time and FLOPs of a single layer of the model on a given input. Appends the results to
  244. the provided list.
  246. Args:
  247. m (nn.Module): The layer to be profiled.
  248. x (torch.Tensor): The input data to the layer.
  249. dt (list): A list to store the computation time of the layer.
  251. Returns:
  252. None
  253. """
  254. if type(x) is tuple:
  255. x = list(x)
  256. c = m == self.model[-1] # is final layer, copy input as inplace fix
  257. if type(x) is list:
  258. try:
  259. bs = x[0].size(0)
  260. except:
  261. bs = x[0][0].size(0)
  262. else:
  263. bs = x.size(0)
  264. o = thop.profile(m, inputs=[x.copy() if c else x], verbose=False)[0] / 1E9 * 2 / bs if thop else 0 # FLOPs
  265. t = time_sync()
  266. for _ in range(10):
  267. m(x.copy() if c else x)
  268. dt.append((time_sync() - t) * 100)
  269. if m == self.model[0]:
  270. LOGGER.info(f"{'time (ms)':>10s} {'GFLOPs':>10s} {'params':>10s} module")
  271. LOGGER.info(f'{dt[-1]:10.2f} {o:10.2f} {get_num_params(m):10.0f} {m.type}')
  272. if c:
  273. LOGGER.info(f"{sum(dt):10.2f} {'-':>10s} {'-':>10s} Total")
  275. def fuse(self, verbose=True):
  276. """
  277. Fuse the `Conv2d()` and `BatchNorm2d()` layers of the model into a single layer, in order to improve the
  278. computation efficiency.
  280. Returns:
  281. (nn.Module): The fused model is returned.
  282. """
  283. if not self.is_fused():
  284. for m in self.model.modules():
  285. if isinstance(m, (Conv, Conv2, DWConv)) and hasattr(m, "bn"):
  286. if isinstance(m, Conv2):
  287. m.fuse_convs()
  288. m.conv = fuse_conv_and_bn(m.conv, m.bn) # update conv
  289. delattr(m, "bn") # remove batchnorm
  290. m.forward = m.forward_fuse # update forward
  291. if isinstance(m, ConvTranspose) and hasattr(m, "bn"):
  292. m.conv_transpose = fuse_deconv_and_bn(m.conv_transpose, m.bn)
  293. delattr(m, "bn") # remove batchnorm
  294. m.forward = m.forward_fuse # update forward
  295. if isinstance(m, RepConv):
  296. m.fuse_convs()
  297. m.forward = m.forward_fuse # update forward
  298. if isinstance(m, RepVGGDW):
  299. m.fuse()
  300. m.forward = m.forward_fuse
  301. if hasattr(m, 'switch_to_deploy'):
  302. m.switch_to_deploy()
  303. self.info(verbose=verbose)
  305. return self
  307. def is_fused(self, thresh=10):
  308. """
  309. Check if the model has less than a certain threshold of BatchNorm layers.
  311. Args:
  312. thresh (int, optional): The threshold number of BatchNorm layers. Default is 10.
  314. Returns:
  315. (bool): True if the number of BatchNorm layers in the model is less than the threshold, False otherwise.
  316. """
  317. bn = tuple(v for k, v in nn.__dict__.items() if "Norm" in k) # normalization layers, i.e. BatchNorm2d()
  318. return sum(isinstance(v, bn) for v in self.modules()) < thresh # True if < 'thresh' BatchNorm layers in model
  320. def info(self, detailed=False, verbose=True, imgsz=640):
  321. """
  322. Prints model information.
  324. Args:
  325. detailed (bool): if True, prints out detailed information about the model. Defaults to False
  326. verbose (bool): if True, prints out the model information. Defaults to False
  327. imgsz (int): the size of the image that the model will be trained on. Defaults to 640
  328. """
  329. return model_info(self, detailed=detailed, verbose=verbose, imgsz=imgsz)
  331. def _apply(self, fn):
  332. """
  333. Applies a function to all the tensors in the model that are not parameters or registered buffers.
  335. Args:
  336. fn (function): the function to apply to the model
  338. Returns:
  339. (BaseModel): An updated BaseModel object.
  340. """
  341. self = super()._apply(fn)
  342. m = self.model[-1] # Detect()
  343. if isinstance(m, DETECT_CLASS): # includes all Detect subclasses like Segment, Pose, OBB, WorldDetect
  344. m.stride = fn(m.stride)
  345. m.anchors = fn(m.anchors)
  346. m.strides = fn(m.strides)
  347. return self
  349. def load(self, weights, verbose=True):
  350. """
  351. Load the weights into the model.
  353. Args:
  354. weights (dict | torch.nn.Module): The pre-trained weights to be loaded.
  355. verbose (bool, optional): Whether to log the transfer progress. Defaults to True.
  356. """
  357. model = weights["model"] if isinstance(weights, dict) else weights # torchvision models are not dicts
  358. csd = model.float().state_dict() # checkpoint state_dict as FP32
  359. csd = intersect_dicts(csd, self.state_dict()) # intersect
  360. self.load_state_dict(csd, strict=False) # load
  361. if verbose:
  362. LOGGER.info(f"Transferred {len(csd)}/{len(self.model.state_dict())} items from pretrained weights")
  364. def loss(self, batch, preds=None):
  365. """
  366. Compute loss.
  368. Args:
  369. batch (dict): Batch to compute loss on
  370. preds (torch.Tensor | List[torch.Tensor]): Predictions.
  371. """
  372. if getattr(self, "criterion", None) is None:
  373. self.criterion = self.init_criterion()
  375. preds = self.forward(batch["img"]) if preds is None else preds
  376. return self.criterion(preds, batch)
  378. def init_criterion(self):
  379. """Initialize the loss criterion for the BaseModel."""
  380. raise NotImplementedError("compute_loss() needs to be implemented by task heads")
  383. class DetectionModel(BaseModel):
  384. """YOLOv8 detection model."""
  386. def __init__(self, cfg="yolov8n.yaml", ch=3, nc=None, verbose=True): # model, input channels, number of classes
  387. """Initialize the YOLOv8 detection model with the given config and parameters."""
  388. super().__init__()
  389. self.yaml = cfg if isinstance(cfg, dict) else yaml_model_load(cfg) # cfg dict
  390. if self.yaml["backbone"][0][2] == "Silence":
  391. LOGGER.warning(
  392. "WARNING ⚠️ YOLOv9 `Silence` module is deprecated in favor of nn.Identity. "
  393. "Please delete local *.pt file and re-download the latest model checkpoint."
  394. )
  395. self.yaml["backbone"][0][2] = "nn.Identity"
  397. # Warehouse_Manager
  398. warehouse_manager_flag = self.yaml.get('Warehouse_Manager', False)
  399. self.warehouse_manager = None
  400. if warehouse_manager_flag:
  401. self.warehouse_manager = Warehouse_Manager(cell_num_ratio=self.yaml.get('Warehouse_Manager_Ratio', 1.0))
  403. # Define model
  404. ch = self.yaml["ch"] = self.yaml.get("ch", ch) # input channels
  405. if nc and nc != self.yaml["nc"]:
  406. LOGGER.info(f"Overriding model.yaml nc={self.yaml['nc']} with nc={nc}")
  407. self.yaml["nc"] = nc # override YAML value
  408. self.model, self.save = parse_model(deepcopy(self.yaml), ch=ch, verbose=verbose, warehouse_manager=self.warehouse_manager) # model, savelist
  409. self.names = {i: f"{i}" for i in range(self.yaml["nc"])} # default names dict
  410. self.inplace = self.yaml.get("inplace", True)
  411. self.end2end = getattr(self.model[-1], "end2end", False)
  412. self.fgi = getattr(self.model[-1], "FGIF", False)
  414. if warehouse_manager_flag:
  415. self.warehouse_manager.store()
  416. self.warehouse_manager.allocate(self)
  417. self.net_update_temperature(0)
  419. # Build strides
  420. m = self.model[-1] # Detect()
  421. if isinstance(m, (DETECT_CLASS + SEGMENT_CLASS + POSE_CLASS + OBB_CLASS + V10_DETECT_CLASS)): # includes all Detect subclasses like Segment, Pose, OBB, WorldDetect
  422. s = 640 # 2x min stride
  423. m.inplace = self.inplace
  425. def _forward(x):
  426. """Performs a forward pass through the model, handling different Detect subclass types accordingly."""
  427. if isinstance(m, (DetectAux,)):
  428. return self.forward(x)[:3]
  429. if self.end2end:
  430. if self.fgi:
  431. return self.forward(x)["output_2"]
  432. else:
  433. return self.forward(x)["one2many"]
  434. return self.forward(x)[0] if isinstance(m, SEGMENT_CLASS + POSE_CLASS + OBB_CLASS) else self.forward(x)
  436. try:
  437. m.stride = torch.tensor([s / x.shape[-2] for x in _forward(torch.zeros(2, ch, s, s))]) # forward
  438. except (RuntimeError, ValueError) as e:
  439. if 'Not implemented on the CPU' in str(e) or 'Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor)' in str(e) or \
  440. 'CUDA tensor' in str(e) or 'is_cuda()' in str(e) or 'carafe_forward_impl' in str(e) or 'Pointer argument (at 0) cannot be accessed from Triton (cpu tensor?)' in str(e):
  441. self.model.to(torch.device('cuda'))
  442. m.stride = torch.tensor([s / x.shape[-2] for x in _forward(torch.zeros(2, ch, s, s).to(torch.device('cuda')))]) # forward
  443. else:
  444. raise e
  445. self.stride = m.stride
  446. m.bias_init() # only run once
  447. else:
  448. self.stride = torch.Tensor([32]) # default stride for i.e. RTDETR
  450. # Init weights, biases
  451. initialize_weights(self)
  452. if verbose:
  453. self.info()
  454. LOGGER.info("")
  456. def _predict_augment(self, x):
  457. """Perform augmentations on input image x and return augmented inference and train outputs."""
  458. if getattr(self, "end2end", False) or self.__class__.__name__ != "DetectionModel":
  459. LOGGER.warning("WARNING ⚠️ Model does not support 'augment=True', reverting to single-scale prediction.")
  460. return self._predict_once(x)
  461. img_size = x.shape[-2:] # height, width
  462. s = [1, 0.83, 0.67] # scales
  463. f = [None, 3, None] # flips (2-ud, 3-lr)
  464. y = [] # outputs
  465. for si, fi in zip(s, f):
  466. xi = scale_img(x.flip(fi) if fi else x, si, gs=int(self.stride.max()))
  467. yi = super().predict(xi)[0] # forward
  468. yi = self._descale_pred(yi, fi, si, img_size)
  469. y.append(yi)
  470. y = self._clip_augmented(y) # clip augmented tails
  471. return torch.cat(y, -1), None # augmented inference, train
  473. @staticmethod
  474. def _descale_pred(p, flips, scale, img_size, dim=1):
  475. """De-scale predictions following augmented inference (inverse operation)."""
  476. p[:, :4] /= scale # de-scale
  477. x, y, wh, cls = p.split((1, 1, 2, p.shape[dim] - 4), dim)
  478. if flips == 2:
  479. y = img_size[0] - y # de-flip ud
  480. elif flips == 3:
  481. x = img_size[1] - x # de-flip lr
  482. return torch.cat((x, y, wh, cls), dim)
  484. def _clip_augmented(self, y):
  485. """Clip YOLO augmented inference tails."""
  486. nl = self.model[-1].nl # number of detection layers (P3-P5)
  487. g = sum(4**x for x in range(nl)) # grid points
  488. e = 1 # exclude layer count
  489. i = (y[0].shape[-1] // g) * sum(4**x for x in range(e)) # indices
  490. y[0] = y[0][..., :-i] # large
  491. i = (y[-1].shape[-1] // g) * sum(4 ** (nl - 1 - x) for x in range(e)) # indices
  492. y[-1] = y[-1][..., i:] # small
  493. return y
  495. def init_criterion(self):
  496. """Initialize the loss criterion for the DetectionModel."""
  497. # return E2EDetectLoss(self) if getattr(self, "end2end", False) else (E2EFGIDetectLoss(self) if getattr(self, "FGIF", False) else v8DetectionLoss(self))
  498. return E2EFGIDetectLoss(self) if getattr(self, "fgi", False) else (E2EDetectLoss(self) if getattr(self, "end2end", False) else v8DetectionLoss(self))
  500. def net_update_temperature(self, temp):
  501. for m in self.modules():
  502. if hasattr(m, "update_temperature"):
  503. m.update_temperature(temp)
  505. class OBBModel(DetectionModel):
  506. """YOLOv8 Oriented Bounding Box (OBB) model."""
  508. def __init__(self, cfg="yolov8n-obb.yaml", ch=3, nc=None, verbose=True):
  509. """Initialize YOLOv8 OBB model with given config and parameters."""
  510. super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  512. def init_criterion(self):
  513. """Initialize the loss criterion for the model."""
  514. return v8OBBLoss(self)
  517. class SegmentationModel(DetectionModel):
  518. """YOLOv8 segmentation model."""
  520. def __init__(self, cfg="yolov8n-seg.yaml", ch=3, nc=None, verbose=True):
  521. """Initialize YOLOv8 segmentation model with given config and parameters."""
  522. super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  524. def init_criterion(self):
  525. """Initialize the loss criterion for the SegmentationModel."""
  526. return v8SegmentationLoss(self)
  529. class PoseModel(DetectionModel):
  530. """YOLOv8 pose model."""
  532. def __init__(self, cfg="yolov8n-pose.yaml", ch=3, nc=None, data_kpt_shape=(None, None), verbose=True):
  533. """Initialize YOLOv8 Pose model."""
  534. if not isinstance(cfg, dict):
  535. cfg = yaml_model_load(cfg) # load model YAML
  536. if any(data_kpt_shape) and list(data_kpt_shape) != list(cfg["kpt_shape"]):
  537. LOGGER.info(f"Overriding model.yaml kpt_shape={cfg['kpt_shape']} with kpt_shape={data_kpt_shape}")
  538. cfg["kpt_shape"] = data_kpt_shape
  539. super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  541. def init_criterion(self):
  542. """Initialize the loss criterion for the PoseModel."""
  543. return v8PoseLoss(self)
  546. class ClassificationModel(BaseModel):
  547. """YOLOv8 classification model."""
  549. def __init__(self, cfg="yolov8n-cls.yaml", ch=3, nc=None, verbose=True):
  550. """Init ClassificationModel with YAML, channels, number of classes, verbose flag."""
  551. super().__init__()
  552. self._from_yaml(cfg, ch, nc, verbose)
  554. def _from_yaml(self, cfg, ch, nc, verbose):
  555. """Set YOLOv8 model configurations and define the model architecture."""
  556. self.yaml = cfg if isinstance(cfg, dict) else yaml_model_load(cfg) # cfg dict
  558. # Define model
  559. ch = self.yaml["ch"] = self.yaml.get("ch", ch) # input channels
  560. if nc and nc != self.yaml["nc"]:
  561. LOGGER.info(f"Overriding model.yaml nc={self.yaml['nc']} with nc={nc}")
  562. self.yaml["nc"] = nc # override YAML value
  563. elif not nc and not self.yaml.get("nc", None):
  564. raise ValueError("nc not specified. Must specify nc in model.yaml or function arguments.")
  565. self.model, self.save = parse_model(deepcopy(self.yaml), ch=ch, verbose=verbose) # model, savelist
  566. self.stride = torch.Tensor([1]) # no stride constraints
  567. self.names = {i: f"{i}" for i in range(self.yaml["nc"])} # default names dict
  568. self.info()
  570. @staticmethod
  571. def reshape_outputs(model, nc):
  572. """Update a TorchVision classification model to class count 'n' if required."""
  573. name, m = list((model.model if hasattr(model, "model") else model).named_children())[-1] # last module
  574. if isinstance(m, Classify): # YOLO Classify() head
  575. if m.linear.out_features != nc:
  576. m.linear = nn.Linear(m.linear.in_features, nc)
  577. elif isinstance(m, nn.Linear): # ResNet, EfficientNet
  578. if m.out_features != nc:
  579. setattr(model, name, nn.Linear(m.in_features, nc))
  580. elif isinstance(m, nn.Sequential):
  581. types = [type(x) for x in m]
  582. if nn.Linear in types:
  583. i = len(types) - 1 - types[::-1].index(nn.Linear) # last nn.Linear index
  584. if m[i].out_features != nc:
  585. m[i] = nn.Linear(m[i].in_features, nc)
  586. elif nn.Conv2d in types:
  587. i = len(types) - 1 - types[::-1].index(nn.Conv2d) # last nn.Conv2d index
  588. if m[i].out_channels != nc:
  589. m[i] = nn.Conv2d(m[i].in_channels, nc, m[i].kernel_size, m[i].stride, bias=m[i].bias is not None)
  591. def init_criterion(self):
  592. """Initialize the loss criterion for the ClassificationModel."""
  593. return v8ClassificationLoss()
  596. class RTDETRDetectionModel(DetectionModel):
  597. """
  598. RTDETR (Real-time DEtection and Tracking using Transformers) Detection Model class.
  600. This class is responsible for constructing the RTDETR architecture, defining loss functions, and facilitating both
  601. the training and inference processes. RTDETR is an object detection and tracking model that extends from the
  602. DetectionModel base class.
  604. Attributes:
  605. cfg (str): The configuration file path or preset string. Default is 'rtdetr-l.yaml'.
  606. ch (int): Number of input channels. Default is 3 (RGB).
  607. nc (int, optional): Number of classes for object detection. Default is None.
  608. verbose (bool): Specifies if summary statistics are shown during initialization. Default is True.
  610. Methods:
  611. init_criterion: Initializes the criterion used for loss calculation.
  612. loss: Computes and returns the loss during training.
  613. predict: Performs a forward pass through the network and returns the output.
  614. """
  616. def __init__(self, cfg="rtdetr-l.yaml", ch=3, nc=None, verbose=True):
  617. """
  618. Initialize the RTDETRDetectionModel.
  620. Args:
  621. cfg (str): Configuration file name or path.
  622. ch (int): Number of input channels.
  623. nc (int, optional): Number of classes. Defaults to None.
  624. verbose (bool, optional): Print additional information during initialization. Defaults to True.
  625. """
  626. super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  628. def init_criterion(self):
  629. """Initialize the loss criterion for the RTDETRDetectionModel."""
  630. from ultralytics.models.utils.loss import RTDETRDetectionLoss
  632. return RTDETRDetectionLoss(nc=self.nc, use_vfl=True)
  634. def loss(self, batch, preds=None):
  635. """
  636. Compute the loss for the given batch of data.
  638. Args:
  639. batch (dict): Dictionary containing image and label data.
  640. preds (torch.Tensor, optional): Precomputed model predictions. Defaults to None.
  642. Returns:
  643. (tuple): A tuple containing the total loss and main three losses in a tensor.
  644. """
  645. if not hasattr(self, "criterion"):
  646. self.criterion = self.init_criterion()
  648. img = batch["img"]
  649. # NOTE: preprocess gt_bbox and gt_labels to list.
  650. bs = len(img)
  651. batch_idx = batch["batch_idx"]
  652. gt_groups = [(batch_idx == i).sum().item() for i in range(bs)]
  653. targets = {
  654. "cls": batch["cls"].to(img.device, dtype=torch.long).view(-1),
  655. "bboxes": batch["bboxes"].to(device=img.device),
  656. "batch_idx": batch_idx.to(img.device, dtype=torch.long).view(-1),
  657. "gt_groups": gt_groups,
  658. }
  660. preds = self.predict(img, batch=targets) if preds is None else preds
  661. dec_bboxes, dec_scores, enc_bboxes, enc_scores, dn_meta = preds if self.training else preds[1]
  662. if dn_meta is None:
  663. dn_bboxes, dn_scores = None, None
  664. else:
  665. dn_bboxes, dec_bboxes = torch.split(dec_bboxes, dn_meta["dn_num_split"], dim=2)
  666. dn_scores, dec_scores = torch.split(dec_scores, dn_meta["dn_num_split"], dim=2)
  668. dec_bboxes = torch.cat([enc_bboxes.unsqueeze(0), dec_bboxes]) # (7, bs, 300, 4)
  669. dec_scores = torch.cat([enc_scores.unsqueeze(0), dec_scores])
  671. loss = self.criterion(
  672. (dec_bboxes, dec_scores), targets, dn_bboxes=dn_bboxes, dn_scores=dn_scores, dn_meta=dn_meta
  673. )
  674. # NOTE: There are like 12 losses in RTDETR, backward with all losses but only show the main three losses.
  675. return sum(loss.values()), torch.as_tensor(
  676. [loss[k].detach() for k in ["loss_giou", "loss_class", "loss_bbox"]], device=img.device
  677. )
  679. def predict(self, x, profile=False, visualize=False, batch=None, augment=False, embed=None):
  680. """
  681. Perform a forward pass through the model.
  683. Args:
  684. x (torch.Tensor): The input tensor.
  685. profile (bool, optional): If True, profile the computation time for each layer. Defaults to False.
  686. visualize (bool, optional): If True, save feature maps for visualization. Defaults to False.
  687. batch (dict, optional): Ground truth data for evaluation. Defaults to None.
  688. augment (bool, optional): If True, perform data augmentation during inference. Defaults to False.
  689. embed (list, optional): A list of feature vectors/embeddings to return.
  691. Returns:
  692. (torch.Tensor): Model's output tensor.
  693. """
  694. y, dt, embeddings = [], [], [] # outputs
  695. for m in self.model[:-1]: # except the head part
  696. if m.f != -1: # if not from previous layer
  697. x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f] # from earlier layers
  698. if profile:
  699. self._profile_one_layer(m, x, dt)
  700. x = m(x) # run
  701. y.append(x if m.i in self.save else None) # save output
  702. if visualize:
  703. feature_visualization(x, m.type, m.i, save_dir=visualize)
  704. if embed and m.i in embed:
  705. embeddings.append(nn.functional.adaptive_avg_pool2d(x, (1, 1)).squeeze(-1).squeeze(-1)) # flatten
  706. if m.i == max(embed):
  707. return torch.unbind(torch.cat(embeddings, 1), dim=0)
  708. head = self.model[-1]
  709. x = head([y[j] for j in head.f], batch) # head inference
  710. return x
  713. class WorldModel(DetectionModel):
  714. """YOLOv8 World Model."""
  716. def __init__(self, cfg="yolov8s-world.yaml", ch=3, nc=None, verbose=True):
  717. """Initialize YOLOv8 world model with given config and parameters."""
  718. self.txt_feats = torch.randn(1, nc or 80, 512) # features placeholder
  719. self.clip_model = None # CLIP model placeholder
  720. super().__init__(cfg=cfg, ch=ch, nc=nc, verbose=verbose)
  722. def set_classes(self, text, batch=80, cache_clip_model=True):
  723. """Set classes in advance so that model could do offline-inference without clip model."""
  724. try:
  725. import clip
  726. except ImportError:
  727. check_requirements("git+https://github.com/ultralytics/CLIP.git")
  728. import clip
  730. if (
  731. not getattr(self, "clip_model", None) and cache_clip_model
  732. ): # for backwards compatibility of models lacking clip_model attribute
  733. self.clip_model = clip.load("ViT-B/32")[0]
  734. model = self.clip_model if cache_clip_model else clip.load("ViT-B/32")[0]
  735. device = next(model.parameters()).device
  736. text_token = clip.tokenize(text).to(device)
  737. txt_feats = [model.encode_text(token).detach() for token in text_token.split(batch)]
  738. txt_feats = txt_feats[0] if len(txt_feats) == 1 else torch.cat(txt_feats, dim=0)
  739. txt_feats = txt_feats / txt_feats.norm(p=2, dim=-1, keepdim=True)
  740. self.txt_feats = txt_feats.reshape(-1, len(text), txt_feats.shape[-1])
  741. self.model[-1].nc = len(text)
  743. def predict(self, x, profile=False, visualize=False, txt_feats=None, augment=False, embed=None):
  744. """
  745. Perform a forward pass through the model.
  747. Args:
  748. x (torch.Tensor): The input tensor.
  749. profile (bool, optional): If True, profile the computation time for each layer. Defaults to False.
  750. visualize (bool, optional): If True, save feature maps for visualization. Defaults to False.
  751. txt_feats (torch.Tensor): The text features, use it if it's given. Defaults to None.
  752. augment (bool, optional): If True, perform data augmentation during inference. Defaults to False.
  753. embed (list, optional): A list of feature vectors/embeddings to return.
  755. Returns:
  756. (torch.Tensor): Model's output tensor.
  757. """
  758. txt_feats = (self.txt_feats if txt_feats is None else txt_feats).to(device=x.device, dtype=x.dtype)
  759. if len(txt_feats) != len(x):
  760. txt_feats = txt_feats.repeat(len(x), 1, 1)
  761. ori_txt_feats = txt_feats.clone()
  762. y, dt, embeddings = [], [], [] # outputs
  763. for m in self.model: # except the head part
  764. if m.f != -1: # if not from previous layer
  765. x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f] # from earlier layers
  766. if profile:
  767. self._profile_one_layer(m, x, dt)
  768. if isinstance(m, C2fAttn):
  769. x = m(x, txt_feats)
  770. elif isinstance(m, WorldDetect):
  771. x = m(x, ori_txt_feats)
  772. elif isinstance(m, ImagePoolingAttn):
  773. txt_feats = m(x, txt_feats)
  774. else:
  775. x = m(x) # run
  777. y.append(x if m.i in self.save else None) # save output
  778. if visualize:
  779. feature_visualization(x, m.type, m.i, save_dir=visualize)
  780. if embed and m.i in embed:
  781. embeddings.append(nn.functional.adaptive_avg_pool2d(x, (1, 1)).squeeze(-1).squeeze(-1)) # flatten
  782. if m.i == max(embed):
  783. return torch.unbind(torch.cat(embeddings, 1), dim=0)
  784. return x
  786. def loss(self, batch, preds=None):
  787. """
  788. Compute loss.
  790. Args:
  791. batch (dict): Batch to compute loss on.
  792. preds (torch.Tensor | List[torch.Tensor]): Predictions.
  793. """
  794. if not hasattr(self, "criterion"):
  795. self.criterion = self.init_criterion()
  797. if preds is None:
  798. preds = self.forward(batch["img"], txt_feats=batch["txt_feats"])
  799. return self.criterion(preds, batch)
  802. class Ensemble(nn.ModuleList):
  803. """Ensemble of models."""
  805. def __init__(self):
  806. """Initialize an ensemble of models."""
  807. super().__init__()
  809. def forward(self, x, augment=False, profile=False, visualize=False):
  810. """Function generates the YOLO network's final layer."""
  811. y = [module(x, augment, profile, visualize)[0] for module in self]
  812. # y = torch.stack(y).max(0)[0] # max ensemble
  813. # y = torch.stack(y).mean(0) # mean ensemble
  814. y = torch.cat(y, 2) # nms ensemble, y shape(B, HW, C)
  815. return y, None # inference, train output
  818. # Functions ------------------------------------------------------------------------------------------------------------
  821. @contextlib.contextmanager
  822. def temporary_modules(modules=None, attributes=None):
  823. """
  824. Context manager for temporarily adding or modifying modules in Python's module cache (`sys.modules`).
  826. This function can be used to change the module paths during runtime. It's useful when refactoring code,
  827. where you've moved a module from one location to another, but you still want to support the old import
  828. paths for backwards compatibility.
  830. Args:
  831. modules (dict, optional): A dictionary mapping old module paths to new module paths.
  832. attributes (dict, optional): A dictionary mapping old module attributes to new module attributes.
  834. Example:
  835. ```python
  836. with temporary_modules({"old.module": "new.module"}, {"old.module.attribute": "new.module.attribute"}):
  837. import old.module # this will now import new.module
  838. from old.module import attribute # this will now import new.module.attribute
  839. ```
  841. Note:
  842. The changes are only in effect inside the context manager and are undone once the context manager exits.
  843. Be aware that directly manipulating `sys.modules` can lead to unpredictable results, especially in larger
  844. applications or libraries. Use this function with caution.
  845. """
  846. if modules is None:
  847. modules = {}
  848. if attributes is None:
  849. attributes = {}
  850. import sys
  851. from importlib import import_module
  853. try:
  854. # Set attributes in sys.modules under their old name
  855. for old, new in attributes.items():
  856. old_module, old_attr = old.rsplit(".", 1)
  857. new_module, new_attr = new.rsplit(".", 1)
  858. setattr(import_module(old_module), old_attr, getattr(import_module(new_module), new_attr))
  860. # Set modules in sys.modules under their old name
  861. for old, new in modules.items():
  862. sys.modules[old] = import_module(new)
  864. yield
  865. finally:
  866. # Remove the temporary module paths
  867. for old in modules:
  868. if old in sys.modules:
  869. del sys.modules[old]
  872. class SafeClass:
  873. """A placeholder class to replace unknown classes during unpickling."""
  875. def __init__(self, *args, **kwargs):
  876. """Initialize SafeClass instance, ignoring all arguments."""
  877. pass
  879. def __call__(self, *args, **kwargs):
  880. """Run SafeClass instance, ignoring all arguments."""
  881. pass
  884. class SafeUnpickler(pickle.Unpickler):
  885. """Custom Unpickler that replaces unknown classes with SafeClass."""
  887. def find_class(self, module, name):
  888. """Attempt to find a class, returning SafeClass if not among safe modules."""
  889. safe_modules = (
  890. "torch",
  891. "collections",
  892. "collections.abc",
  893. "builtins",
  894. "math",
  895. "numpy",
  896. # Add other modules considered safe
  897. )
  898. if module in safe_modules:
  899. return super().find_class(module, name)
  900. else:
  901. return SafeClass
  904. def torch_safe_load(weight, safe_only=False):
  905. """
  906. Attempts to load a PyTorch model with the torch.load() function. If a ModuleNotFoundError is raised, it catches the
  907. error, logs a warning message, and attempts to install the missing module via the check_requirements() function.
  908. After installation, the function again attempts to load the model using torch.load().
  910. Args:
  911. weight (str): The file path of the PyTorch model.
  912. safe_only (bool): If True, replace unknown classes with SafeClass during loading.
  914. Example:
  915. ```python
  916. from ultralytics.nn.tasks import torch_safe_load
  918. ckpt, file = torch_safe_load("path/to/best.pt", safe_only=True)
  919. ```
  921. Returns:
  922. ckpt (dict): The loaded model checkpoint.
  923. file (str): The loaded filename
  924. """
  925. from ultralytics.utils.downloads import attempt_download_asset
  927. check_suffix(file=weight, suffix=".pt")
  928. file = attempt_download_asset(weight) # search online if missing locally
  929. try:
  930. with temporary_modules(
  931. modules={
  932. "ultralytics.yolo.utils": "ultralytics.utils",
  933. "ultralytics.yolo.v8": "ultralytics.models.yolo",
  934. "ultralytics.yolo.data": "ultralytics.data",
  935. },
  936. attributes={
  937. "ultralytics.nn.modules.block.Silence": "torch.nn.Identity", # YOLOv9e
  938. "ultralytics.nn.tasks.YOLOv10DetectionModel": "ultralytics.nn.tasks.DetectionModel", # YOLOv10
  939. "ultralytics.utils.loss.v10DetectLoss": "ultralytics.utils.loss.E2EDetectLoss", # YOLOv10
  940. },
  941. ):
  942. if safe_only:
  943. # Load via custom pickle module
  944. safe_pickle = types.ModuleType("safe_pickle")
  945. safe_pickle.Unpickler = SafeUnpickler
  946. safe_pickle.load = lambda file_obj: SafeUnpickler(file_obj).load()
  947. with open(file, "rb") as f:
  948. ckpt = torch.load(f, pickle_module=safe_pickle)
  949. else:
  950. ckpt = torch.load(file, map_location="cpu")
  952. except ModuleNotFoundError as e: # e.name is missing module name
  953. if e.name == "models":
  954. raise TypeError(
  955. emojis(
  956. f"ERROR ❌️ {weight} appears to be an Ultralytics YOLOv5 model originally trained "
  957. f"with https://github.com/ultralytics/yolov5.\nThis model is NOT forwards compatible with "
  958. f"YOLOv8 at https://github.com/ultralytics/ultralytics."
  959. f"\nRecommend fixes are to train a new model using the latest 'ultralytics' package or to "
  960. f"run a command with an official Ultralytics model, i.e. 'yolo predict model=yolov8n.pt'"
  961. )
  962. ) from e
  963. LOGGER.warning(
  964. f"WARNING ⚠️ {weight} appears to require '{e.name}', which is not in Ultralytics requirements."
  965. f"\nAutoInstall will run now for '{e.name}' but this feature will be removed in the future."
  966. f"\nRecommend fixes are to train a new model using the latest 'ultralytics' package or to "
  967. f"run a command with an official Ultralytics model, i.e. 'yolo predict model=yolov8n.pt'"
  968. )
  969. check_requirements(e.name) # install missing module
  970. ckpt = torch.load(file, map_location="cpu")
  972. if not isinstance(ckpt, dict):
  973. # File is likely a YOLO instance saved with i.e. torch.save(model, "saved_model.pt")
  974. LOGGER.warning(
  975. f"WARNING ⚠️ The file '{weight}' appears to be improperly saved or formatted. "
  976. f"For optimal results, use model.save('filename.pt') to correctly save YOLO models."
  977. )
  978. ckpt = {"model": ckpt.model}
  980. return ckpt, file
  983. def attempt_load_weights(weights, device=None, inplace=True, fuse=False):
  984. """Loads an ensemble of models weights=[a,b,c] or a single model weights=[a] or weights=a."""
  985. ensemble = Ensemble()
  986. for w in weights if isinstance(weights, list) else [weights]:
  987. ckpt, w = torch_safe_load(w) # load ckpt
  988. args = {**DEFAULT_CFG_DICT, **ckpt["train_args"]} if "train_args" in ckpt else None # combined args
  989. model = (ckpt.get("ema") or ckpt["model"]).to(device).float() # FP32 model
  991. # Model compatibility updates
  992. model.args = args # attach args to model
  993. model.pt_path = w # attach *.pt file path to model
  994. model.task = guess_model_task(model)
  995. if not hasattr(model, "stride"):
  996. model.stride = torch.tensor([32.0])
  998. # Append
  999. ensemble.append(model.fuse().eval() if fuse and hasattr(model, "fuse") else model.eval()) # model in eval mode
  1001. # Module updates
  1002. for m in ensemble.modules():
  1003. if hasattr(m, "inplace"):
  1004. m.inplace = inplace
  1005. elif isinstance(m, nn.Upsample) and not hasattr(m, "recompute_scale_factor"):
  1006. m.recompute_scale_factor = None # torch 1.11.0 compatibility
  1008. # Return model
  1009. if len(ensemble) == 1:
  1010. return ensemble[-1]
  1012. # Return ensemble
  1013. LOGGER.info(f"Ensemble created with {weights}\n")
  1014. for k in "names", "nc", "yaml":
  1015. setattr(ensemble, k, getattr(ensemble[0], k))
  1016. ensemble.stride = ensemble[int(torch.argmax(torch.tensor([m.stride.max() for m in ensemble])))].stride
  1017. assert all(ensemble[0].nc == m.nc for m in ensemble), f"Models differ in class counts {[m.nc for m in ensemble]}"
  1018. return ensemble
  1021. def attempt_load_one_weight(weight, device=None, inplace=True, fuse=False):
  1022. """Loads a single model weights."""
  1023. ckpt, weight = torch_safe_load(weight) # load ckpt
  1024. args = {**DEFAULT_CFG_DICT, **(ckpt.get("train_args", {}))} # combine model and default args, preferring model args
  1025. model = (ckpt.get("ema") or ckpt["model"]).to(device).float() # FP32 model
  1027. # Model compatibility updates
  1028. model.args = {k: v for k, v in args.items() if k in DEFAULT_CFG_KEYS} # attach args to model
  1029. model.pt_path = weight # attach *.pt file path to model
  1030. model.task = guess_model_task(model)
  1031. if not hasattr(model, "stride"):
  1032. model.stride = torch.tensor([32.0])
  1034. model = model.fuse().eval() if fuse and hasattr(model, "fuse") else model.eval() # model in eval mode
  1036. # Module updates
  1037. for m in model.modules():
  1038. if hasattr(m, "inplace"):
  1039. m.inplace = inplace
  1040. elif isinstance(m, nn.Upsample) and not hasattr(m, "recompute_scale_factor"):
  1041. m.recompute_scale_factor = None # torch 1.11.0 compatibility
  1043. # Return model and ckpt
  1044. return model, ckpt
  1047. def parse_model(d, ch, verbose=True, warehouse_manager=None): # model_dict, input_channels(3)
  1048. """Parse a YOLO model.yaml dictionary into a PyTorch model."""
  1049. import ast
  1051. # Args
  1052. max_channels = float("inf")
  1053. nc, act, scales = (d.get(x) for x in ("nc", "activation", "scales"))
  1054. depth, width, kpt_shape = (d.get(x, 1.0) for x in ("depth_multiple", "width_multiple", "kpt_shape"))
  1055. if scales:
  1056. scale = d.get("scale")
  1057. if not scale:
  1058. scale = tuple(scales.keys())[0]
  1059. LOGGER.warning(f"WARNING ⚠️ no model scale passed. Assuming scale='{scale}'.")
  1060. if len(scales[scale]) == 3:
  1061. depth, width, max_channels = scales[scale]
  1062. elif len(scales[scale]) == 4:
  1063. depth, width, max_channels, threshold = scales[scale]
  1065. if act:
  1066. Conv.default_act = eval(act) # redefine default activation, i.e. Conv.default_act = nn.SiLU()
  1067. if verbose:
  1068. LOGGER.info(f"{colorstr('activation:')} {act}") # print
  1070. if verbose:
  1071. LOGGER.info(f"\n{'':>3}{'from':>20}{'n':>3}{'params':>10} {'module':<45}{'arguments':<30}")
  1072. ch = [ch]
  1073. layers, save, c2 = [], [], ch[-1] # layers, savelist, ch out
  1074. is_backbone = False
  1075. for i, (f, n, m, args) in enumerate(d["backbone"] + d["head"]): # from, number, module, args
  1076. try:
  1077. if m == 'node_mode':
  1078. m = d[m]
  1079. if len(args) > 0:
  1080. if args[0] == 'head_channel':
  1081. args[0] = int(d[args[0]])
  1082. t = m
  1083. m = getattr(torch.nn, m[3:]) if 'nn.' in m else globals()[m] # get module
  1084. except:
  1085. pass
  1086. for j, a in enumerate(args):
  1087. if isinstance(a, str):
  1088. with contextlib.suppress(ValueError):
  1089. try:
  1090. args[j] = locals()[a] if a in locals() else ast.literal_eval(a)
  1091. except:
  1092. args[j] = a
  1094. n = n_ = max(round(n * depth), 1) if n > 1 else n # depth gain
  1095. if m in ((
  1096. Classify, Conv, ConvTranspose, GhostConv, Bottleneck, GhostBottleneck, SPP, SPPF, C2fPSA, C2PSA, DWConv, Focus, BottleneckCSP,
  1097. C1, C2, C2f, C3k2, RepNCSPELAN4, ELAN1, ADown, AConv, SPPELAN, C2fAttn, C3, C3TR, C3Ghost, nn.ConvTranspose2d, DWConvTranspose2d,
  1098. C3x, RepC3, PSA, SCDown, C2fCIB, GSConv, GSConvns, VoVGSCSP, VoVGSCSPns, VoVGSCSPC, RCSOSA, KWConv, CSPStage, SPDConv, RepBlock,
  1099. SPPF_LSKA, CSP_EDLAN, nn.Conv2d, HWD, RepNCSPELAN4, DBBNCSPELAN4, OREPANCSPELAN4, DRBNCSPELAN4, RepNCSPELAN4_CAA, V7DownSampling,
  1100. DGCST, RepNCSPELAN4_CAA, SRFD, DRFD, RGCSPELAN, CSP_PTB, SimpleStem, VisionClueMerge, VSSBlock_YOLO, XSSBlock, GLSA, FeaturePyramidSharedConv,
  1101. LDConv, CSP_MSCB, CSP_PMSFA, RFAConv, RFCBAMConv, RFCAConv, CSP_FreqSpatial, C2BRA, C2CGA, C2DA, C2DPB, MANet, MANet_FasterBlock, MANet_FasterCGLU,
  1102. MANet_Star, PSConv, C2Pola, A2C2f, C2TSSA, A2C2f_CGLU, A2C2f_KAN, A2C2f_DFFN, RepHMS, A2C2f_FRFN, C2ASSA, C2PSA_DYT, C2TSSA_DYT, C2Pola_DYT,
  1103. A2C2f_DYT, A2C2f_CGLU_DYT, A2C2f_DFFN_DYT
  1104. ) + C3K2_CLASS):
  1105. if args[0] == 'head_channel':
  1106. args[0] = d[args[0]]
  1107. c1, c2 = ch[f], args[0]
  1108. if c2 != nc: # if c2 not equal to number of classes (i.e. for Classify() output)
  1109. c2 = make_divisible(min(c2, max_channels) * width, 8)
  1110. if m is C2fAttn:
  1111. args[1] = make_divisible(min(args[1], max_channels // 2) * width, 8) # embed channels
  1112. args[2] = int(
  1113. max(round(min(args[2], max_channels // 2 // 32)) * width, 1) if args[2] > 1 else args[2]
  1114. ) # num heads
  1116. args = [c1, c2, *args[1:]]
  1117. if m in (KWConv, C3k2_KW):
  1118. args.insert(2, f'layer{i}')
  1119. args.insert(2, warehouse_manager)
  1120. if m in (DySnakeConv,):
  1121. c2 = c2 * 3
  1122. if m in (RepNCSPELAN4, DBBNCSPELAN4, OREPANCSPELAN4, DRBNCSPELAN4, RepNCSPELAN4_CAA):
  1123. args[2] = make_divisible(min(args[2], max_channels) * width, 8)
  1124. args[3] = make_divisible(min(args[3], max_channels) * width, 8)
  1125. if m in ((
  1126. BottleneckCSP, C1, C2, C2f, C3k2, C2fAttn, C3, C3TR, C3Ghost, C3x, RepC3, C2fPSA, C2fCIB, C2PSA, VoVGSCSP, VoVGSCSPns, VoVGSCSPC, RCSOSA,
  1127. CSPStage, SPDConv, RepBlock, CSP_EDLAN, RGCSPELAN, CSP_PTB, XSSBlock, CSP_MSCB, CSP_PMSFA, CSP_FreqSpatial, C2BRA, C2CGA, C2DA, C2DPB, MANet,
  1128. MANet_FasterBlock, MANet_FasterCGLU, MANet_Star, C2Pola, A2C2f, C2TSSA, A2C2f_CGLU, A2C2f_KAN, A2C2f_DFFN, A2C2f_FRFN, C2ASSA, C2PSA_DYT,
  1129. C2TSSA_DYT, C2Pola_DYT, A2C2f_DYT, A2C2f_CGLU_DYT, A2C2f_DFFN_DYT
  1130. ) + C3K2_CLASS):
  1131. args.insert(2, n) # number of repeats
  1132. n = 1
  1133. if m in ((C3k2,) + C3K2_CLASS) and scale in "mlx": # for M/L/X sizes
  1134. if m in (C3k2_AggregatedAtt, C3k2_MSBlock, C3k2_DAttention, C3k2_FocusedLinearAttention, C3k2_Parc, C3k2_UniRepLKNetBlock, C3k2_SMPCGLU,
  1135. C3k2_Heat, C3k2_RandomMixing, C3k2_RandomMixingCGLU, C3k2_PKIModule, C3k2_FCA_CTA, C3k2_GlobalFilter, C3k2_DynamicFilter):
  1136. if type(args[-1]) == bool:
  1137. args[-1] = True
  1138. else:
  1139. args[-2] = True
  1140. else:
  1141. args[3] = True
  1142. if m in (A2C2f,) and scale in "lx": # for L/X sizes
  1143. args.extend((True, 1.2))
  1144. elif m in {AIFI, AIFI_RepBN, DTAB, ETB, FDT}:
  1145. args = [ch[f], *args]
  1146. c2 = args[0]
  1147. elif m in (HGStem, HGBlock, Ghost_HGBlock, Rep_HGBlock, Dynamic_HGBlock, EIEStem):
  1148. c1, cm, c2 = ch[f], args[0], args[1]
  1149. if c2 != nc: # if c2 not equal to number of classes (i.e. for Classify() output)
  1150. c2 = make_divisible(min(c2, max_channels) * width, 8)
  1151. cm = make_divisible(min(cm, max_channels) * width, 8)
  1152. args = [c1, cm, c2, *args[2:]]
  1153. if m in (HGBlock, Ghost_HGBlock, Rep_HGBlock, Dynamic_HGBlock):
  1154. args.insert(4, n) # number of repeats
  1155. n = 1
  1156. elif m is ResNetLayer:
  1157. c2 = args[1] if args[3] else args[1] * 4
  1158. elif m is nn.BatchNorm2d:
  1159. args = [ch[f]]
  1160. elif m is Concat:
  1161. c2 = sum(ch[x] for x in f)
  1162. elif m in ((WorldDetect, ImagePoolingAttn) + DETECT_CLASS + V10_DETECT_CLASS + SEGMENT_CLASS + POSE_CLASS + OBB_CLASS):
  1163. args.append([ch[x] for x in f])
  1164. if m in SEGMENT_CLASS:
  1165. args[2] = make_divisible(min(args[2], max_channels) * width, 8)
  1166. if m in (Segment_LSCD, Segment_TADDH, Segment_LSCSBD, Segment_LSDECD, Segment_RSCD):
  1167. args[3] = make_divisible(min(args[3], max_channels) * width, 8)
  1168. if m in (Detect_LSCD, Detect_TADDH, Detect_LSCSBD, Detect_LSDECD, Detect_RSCD):
  1169. args[1] = make_divisible(min(args[1], max_channels) * width, 8)
  1170. if m in (Pose_LSCD, Pose_TADDH, Pose_LSCSBD, Pose_LSDECD, Pose_RSCD, OBB_LSCD, OBB_TADDH, OBB_LSCSBD, OBB_LSDECD, OBB_RSCD):
  1171. args[2] = make_divisible(min(args[2], max_channels) * width, 8)
  1172. elif m is RTDETRDecoder: # special case, channels arg must be passed in index 1
  1173. args.insert(1, [ch[x] for x in f])
  1174. elif m is Fusion:
  1175. args[0] = d[args[0]]
  1176. c1, c2 = [ch[x] for x in f], (sum([ch[x] for x in f]) if args[0] == 'concat' else ch[f[0]])
  1177. args = [c1, args[0]]
  1178. elif m is CBLinear:
  1179. c2 = make_divisible(min(args[0][-1], max_channels) * width, 8)
  1180. c1 = ch[f]
  1181. args = [c1, [make_divisible(min(c2_, max_channels) * width, 8) for c2_ in args[0]], *args[1:]]
  1182. elif m is CBFuse:
  1183. c2 = ch[f[-1]]
  1184. elif isinstance(m, str):
  1185. t = m
  1186. if len(args) == 2:
  1187. m = timm.create_model(m, pretrained=args[0], pretrained_cfg_overlay={'file':args[1]}, features_only=True)
  1188. elif len(args) == 1:
  1189. m = timm.create_model(m, pretrained=args[0], features_only=True)
  1190. c2 = m.feature_info.channels()
  1191. elif m in {convnextv2_atto, convnextv2_femto, convnextv2_pico, convnextv2_nano, convnextv2_tiny, convnextv2_base, convnextv2_large, convnextv2_huge,
  1192. fasternet_t0, fasternet_t1, fasternet_t2, fasternet_s, fasternet_m, fasternet_l,
  1193. EfficientViT_M0, EfficientViT_M1, EfficientViT_M2, EfficientViT_M3, EfficientViT_M4, EfficientViT_M5,
  1194. efficientformerv2_s0, efficientformerv2_s1, efficientformerv2_s2, efficientformerv2_l,
  1195. vanillanet_5, vanillanet_6, vanillanet_7, vanillanet_8, vanillanet_9, vanillanet_10, vanillanet_11, vanillanet_12, vanillanet_13, vanillanet_13_x1_5, vanillanet_13_x1_5_ada_pool,
  1196. RevCol,
  1197. lsknet_t, lsknet_s,
  1198. SwinTransformer_Tiny,
  1199. repvit_m0_9, repvit_m1_0, repvit_m1_1, repvit_m1_5, repvit_m2_3,
  1200. CSWin_tiny, CSWin_small, CSWin_base, CSWin_large,
  1201. unireplknet_a, unireplknet_f, unireplknet_p, unireplknet_n, unireplknet_t, unireplknet_s, unireplknet_b, unireplknet_l, unireplknet_xl,
  1202. transnext_micro, transnext_tiny, transnext_small, transnext_base,
  1203. RMT_T, RMT_S, RMT_B, RMT_L,
  1204. PKINET_T, PKINET_S, PKINET_B,
  1205. MobileNetV4ConvSmall, MobileNetV4ConvMedium, MobileNetV4ConvLarge, MobileNetV4HybridMedium, MobileNetV4HybridLarge,
  1206. starnet_s050, starnet_s100, starnet_s150, starnet_s1, starnet_s2, starnet_s3, starnet_s4,
  1207. inceptionnext_tiny, inceptionnext_small, inceptionnext_base, inceptionnext_base_384,
  1208. MobileMamba_T2, MobileMamba_T4, MobileMamba_S6, MobileMamba_B1, MobileMamba_B2, MobileMamba_B4,
  1209. mambaout_femto, mambaout_kobe, mambaout_tiny, mambaout_small, mambaout_base
  1210. }:
  1211. if m is RevCol:
  1212. args[1] = [make_divisible(min(k, max_channels) * width, 8) for k in args[1]]
  1213. args[2] = [max(round(k * depth), 1) for k in args[2]]
  1214. m = m(*args)
  1215. c2 = m.channel
  1216. elif m in {EMA, SpatialAttention, BiLevelRoutingAttention, BiLevelRoutingAttention_nchw,
  1217. TripletAttention, CoordAtt, CBAM, BAMBlock, LSKBlock, ScConv, LAWDS, EMSConv, EMSConvP,
  1218. SEAttention, CPCA, Partial_conv3, FocalModulation, EfficientAttention, MPCA, deformable_LKA,
  1219. EffectiveSEModule, LSKA, SegNext_Attention, DAttention, MLCA, TransNeXt_AggregatedAttention,
  1220. FocusedLinearAttention, LocalWindowAttention, ChannelAttention_HSFPN, ELA_HSFPN, CA_HSFPN, CAA_HSFPN,
  1221. DySample, CARAFE, CAA, ELA, CAFM, AFGCAttention, EUCB, EfficientChannelAttention, ContrastDrivenFeatureAggregation,
  1222. FSA, AttentiveLayer}:
  1223. c2 = ch[f]
  1224. args = [c2, *args]
  1225. # print(args)
  1226. elif m in {SimAM, SpatialGroupEnhance}:
  1227. c2 = ch[f]
  1228. args = [*args]
  1229. elif m is ContextGuidedBlock_Down:
  1230. c2 = ch[f] * 2
  1231. args = [ch[f], c2, *args]
  1232. elif m is BiFusion:
  1233. c1 = [ch[x] for x in f]
  1234. c2 = make_divisible(min(args[0], max_channels) * width, 8)
  1235. args = [c1, c2]
  1236. # --------------GOLD-YOLO--------------
  1237. elif m in {SimFusion_4in, AdvPoolFusion}:
  1238. c2 = sum(ch[x] for x in f)
  1239. elif m is SimFusion_3in:
  1240. c2 = args[0]
  1241. if c2 != nc: # if c2 not equal to number of classes (i.e. for Classify() output)
  1242. c2 = make_divisible(min(c2, max_channels) * width, 8)
  1243. args = [[ch[f_] for f_ in f], c2]
  1244. elif m is IFM:
  1245. c1 = ch[f]
  1246. c2 = sum(args[0])
  1247. args = [c1, *args]
  1248. elif m is InjectionMultiSum_Auto_pool:
  1249. c1 = ch[f[0]]
  1250. c2 = args[0]
  1251. args = [c1, *args]
  1252. elif m is PyramidPoolAgg:
  1253. c2 = args[0]
  1254. args = [sum([ch[f_] for f_ in f]), *args]
  1255. elif m is TopBasicLayer:
  1256. c2 = sum(args[1])
  1257. # --------------GOLD-YOLO--------------
  1258. # --------------ASF--------------
  1259. elif m is Zoom_cat:
  1260. c2 = sum(ch[x] for x in f)
  1261. elif m is Add:
  1262. c2 = ch[f[-1]]
  1263. elif m in {ScalSeq, DynamicScalSeq}:
  1264. c1 = [ch[x] for x in f]
  1265. c2 = make_divisible(args[0] * width, 8)
  1266. args = [c1, c2]
  1267. elif m is asf_attention_model:
  1268. args = [ch[f[-1]]]
  1269. # --------------ASF--------------
  1270. elif m is SDI:
  1271. args = [[ch[x] for x in f]]
  1272. elif m is Multiply:
  1273. c2 = ch[f[0]]
  1274. elif m is FocusFeature:
  1275. c1 = [ch[x] for x in f]
  1276. c2 = int(c1[1] * 0.5 * 3)
  1277. args = [c1, *args]
  1278. elif m is DASI:
  1279. c1 = [ch[x] for x in f]
  1280. args = [c1, c2]
  1281. elif m is CSMHSA:
  1282. c1 = [ch[x] for x in f]
  1283. c2 = ch[f[-1]]
  1284. args = [c1, c2]
  1285. elif m is CFC_CRB:
  1286. c1 = ch[f]
  1287. c2 = c1 // 2
  1288. args = [c1, *args]
  1289. elif m is SFC_G2:
  1290. c1 = [ch[x] for x in f]
  1291. c2 = c1[0]
  1292. args = [c1]
  1293. elif m in {CGAFusion, CAFMFusion, SDFM, PSFM}:
  1294. c2 = ch[f[1]]
  1295. args = [c2, *args]
  1296. elif m in {ContextGuideFusionModule}:
  1297. c1 = [ch[x] for x in f]
  1298. c2 = 2 * c1[1]
  1299. args = [c1]
  1300. # elif m in {PSA}:
  1301. # c2 = ch[f]
  1302. # args = [c2, *args]
  1303. elif m in {SBA}:
  1304. c1 = [ch[x] for x in f]
  1305. c2 = c1[-1]
  1306. args = [c1, c2]
  1307. elif m in {WaveletPool}:
  1308. c2 = ch[f] * 4
  1309. elif m in {WaveletUnPool}:
  1310. c2 = ch[f] // 4
  1311. elif m in {CSPOmniKernel}:
  1312. c2 = ch[f]
  1313. args = [c2]
  1314. elif m in {ChannelTransformer, PyramidContextExtraction}:
  1315. c1 = [ch[x] for x in f]
  1316. c2 = c1
  1317. args = [c1]
  1318. elif m in {RCM}:
  1319. c2 = ch[f]
  1320. args = [c2, *args]
  1321. elif m in {DynamicInterpolationFusion}:
  1322. c2 = ch[f[0]]
  1323. args = [[ch[x] for x in f]]
  1324. elif m in {FuseBlockMulti}:
  1325. c2 = ch[f[0]]
  1326. args = [c2]
  1327. elif m in {CrossLayerChannelAttention, CrossLayerSpatialAttention}:
  1328. c2 = [ch[x] for x in f]
  1329. args = [c2[0], *args]
  1330. elif m in {FreqFusion}:
  1331. c2 = ch[f[0]]
  1332. args = [[ch[x] for x in f], *args]
  1333. elif m in {DynamicAlignFusion}:
  1334. c2 = args[0]
  1335. args = [[ch[x] for x in f], c2]
  1336. elif m in {ConvEdgeFusion}:
  1337. c2 = make_divisible(min(args[0], max_channels) * width, 8)
  1338. args = [[ch[x] for x in f], c2]
  1339. elif m in {MutilScaleEdgeInfoGenetator}:
  1340. c1 = ch[f]
  1341. c2 = [make_divisible(min(i, max_channels) * width, 8) for i in args[0]]
  1342. args = [c1, c2]
  1343. elif m in {MultiScaleGatedAttn}:
  1344. c1 = [ch[x] for x in f]
  1345. c2 = min(c1)
  1346. args = [c1]
  1347. elif m is HyperComputeModule:
  1348. c1, c2 = ch[f], args[0]
  1349. c2 = make_divisible(min(c2, max_channels) * width, 8)
  1350. args = [c1, c2, threshold]
  1351. elif m in {WFU, MultiScalePCA, MultiScalePCA_Down}:
  1352. c1 = [ch[x] for x in f]
  1353. c2 = c1[0]
  1354. args = [c1]
  1355. elif m in {HAFB}:
  1356. c1 = [ch[x] for x in f]
  1357. c2 = make_divisible(min(args[0], max_channels) * width, 8)
  1358. args = [c1, c2, *args[1:]]
  1359. elif m in {GetIndexOutput}:
  1360. c2 = ch[f][args[0]]
  1361. else:
  1362. c2 = ch[f]
  1364. if isinstance(c2, list) and m not in {ChannelTransformer, PyramidContextExtraction, CrossLayerChannelAttention, CrossLayerSpatialAttention, MutilScaleEdgeInfoGenetator}:
  1365. is_backbone = True
  1366. m_ = m
  1367. m_.backbone = True
  1368. else:
  1369. m_ = nn.Sequential(*(m(*args) for _ in range(n))) if n > 1 else m(*args) # module
  1370. t = str(m)[8:-2].replace('__main__.', '') # module type
  1371. m_.np = sum(x.numel() for x in m_.parameters()) # number params
  1372. m_.i, m_.f, m_.type = i + 4 if is_backbone else i, f, t # attach index, 'from' index, type
  1373. if verbose:
  1374. LOGGER.info(f"{i:>3}{str(f):>20}{n_:>3}{m_.np:10.0f} {t:<45}{str(args):<30}") # print
  1375. save.extend(x % (i + 4 if is_backbone else i) for x in ([f] if isinstance(f, int) else f) if x != -1) # append to savelist
  1376. layers.append(m_)
  1377. if i == 0:
  1378. ch = []
  1379. if isinstance(c2, list) and m not in {ChannelTransformer, PyramidContextExtraction, CrossLayerChannelAttention, CrossLayerSpatialAttention, MutilScaleEdgeInfoGenetator}:
  1380. ch.extend(c2)
  1381. for _ in range(5 - len(ch)):
  1382. ch.insert(0, 0)
  1383. else:
  1384. ch.append(c2)
  1385. return nn.Sequential(*layers), sorted(save)
  1388. def yaml_model_load(path):
  1389. """Load a YOLOv8 model from a YAML file."""
  1390. path = Path(path)
  1391. if path.stem in (f"yolov{d}{x}6" for x in "nsmlx" for d in (5, 8)):
  1392. new_stem = re.sub(r"(\d+)([nslmx])6(.+)?$", r"\1\2-p6\3", path.stem)
  1393. LOGGER.warning(f"WARNING ⚠️ Ultralytics YOLO P6 models now use -p6 suffix. Renaming {path.stem} to {new_stem}.")
  1394. path = path.with_name(new_stem + path.suffix)
  1396. unified_path = re.sub(r"(\d+)([nslmx])(.+)?$", r"\1\3", str(path)) # i.e. yolov8x.yaml -> yolov8.yaml
  1397. unified_path = re.sub(r'(hyper-yolo)([nslmx])(.+)?$', r'\1\3', str(unified_path))
  1398. unified_path = re.sub(r'(hyper-yolot)(.+)?$', r'\1\2', str(unified_path))
  1399. yaml_file = check_yaml(unified_path, hard=False) or check_yaml(path)
  1400. d = yaml_load(yaml_file) # model dict
  1401. d["scale"] = guess_model_scale(path)
  1402. d["yaml_file"] = str(path)
  1403. return d
  1406. def guess_model_scale(model_path):
  1407. """
  1408. Takes a path to a YOLO model's YAML file as input and extracts the size character of the model's scale. The function
  1409. uses regular expression matching to find the pattern of the model scale in the YAML file name, which is denoted by
  1410. n, s, m, l, or x. The function returns the size character of the model scale as a string.
  1412. Args:
  1413. model_path (str | Path): The path to the YOLO model's YAML file.
  1415. Returns:
  1416. (str): The size character of the model's scale, which can be n, s, m, l, or x.
  1417. """
  1418. try:
  1419. if re.search(r'(hyper-yolo)([tnslmx])', Path(model_path).stem) is not None:
  1420. return re.search(r'(hyper-yolo)([tnslmx])', Path(model_path).stem).group(2)
  1421. else:
  1422. return re.search(r"yolo[v]?\d+([nslmx])", Path(model_path).stem).group(1) # n, s, m, l, or x
  1423. except AttributeError:
  1424. return ""
  1427. def guess_model_task(model):
  1428. """
  1429. Guess the task of a PyTorch model from its architecture or configuration.
  1431. Args:
  1432. model (nn.Module | dict): PyTorch model or model configuration in YAML format.
  1434. Returns:
  1435. (str): Task of the model ('detect', 'segment', 'classify', 'pose').
  1437. Raises:
  1438. SyntaxError: If the task of the model could not be determined.
  1439. """
  1441. def cfg2task(cfg):
  1442. """Guess from YAML dictionary."""
  1443. m = cfg["head"][-1][-2].lower() # output module name
  1444. if m in {"classify", "classifier", "cls", "fc"}:
  1445. return "classify"
  1446. if "detect" in m:
  1447. return "detect"
  1448. if "segment" in m:
  1449. return "segment"
  1450. if "pose" in m:
  1451. return "pose"
  1452. if "obb" in m:
  1453. return "obb"
  1455. # Guess from model cfg
  1456. if isinstance(model, dict):
  1457. try:
  1458. return cfg2task(model)
  1459. except: # noqa E722
  1460. pass
  1462. # Guess from PyTorch model
  1463. if isinstance(model, nn.Module): # PyTorch model
  1464. for x in "model.args", "model.model.args", "model.model.model.args":
  1465. try:
  1466. return eval(x)["task"]
  1467. except: # noqa E722
  1468. pass
  1469. for x in "model.yaml", "model.model.yaml", "model.model.model.yaml":
  1470. try:
  1471. return cfg2task(eval(x))
  1472. except: # noqa E722
  1473. pass
  1475. for m in model.modules():
  1476. if isinstance(m, Segment):
  1477. return "segment"
  1478. elif isinstance(m, Classify):
  1479. return "classify"
  1480. elif isinstance(m, Pose):
  1481. return "pose"
  1482. elif isinstance(m, OBB):
  1483. return "obb"
  1484. elif isinstance(m, (Detect, WorldDetect, v10Detect)):
  1485. return "detect"
  1487. # Guess from model filename
  1488. if isinstance(model, (str, Path)):
  1489. model = Path(model)
  1490. if "-seg" in model.stem or "segment" in model.parts:
  1491. return "segment"
  1492. elif "-cls" in model.stem or "classify" in model.parts:
  1493. return "classify"
  1494. elif "-pose" in model.stem or "pose" in model.parts:
  1495. return "pose"
  1496. elif "-obb" in model.stem or "obb" in model.parts:
  1497. return "obb"
  1498. elif "detect" in model.parts:
  1499. return "detect"
  1501. # Unable to determine task from model
  1502. LOGGER.warning(
  1503. "WARNING ⚠️ Unable to automatically guess model task, assuming 'task=detect'. "
  1504. "Explicitly define task for your model, i.e. 'task=detect', 'segment', 'classify','pose' or 'obb'."
  1505. )
  1506. return "detect" # assume detect

tasks.py at commit 680f06b, no license · at the source

Overview

Authors: Ronghui Zheng1, Shanshan Cai2
ORCID iDs: Shanshan Cai
  1. Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, China
  2. Division of Biomedical and Life Sciences, Faculty of Health and Medicine, Lancaster University, UK
Institutions: Fujian Medical University (China); Fujian Provincial Cancer Hospital (China); Lancaster University (United Kingdom)
Journal: The Journal of international medical research, volume 54, issue 3, article 03000605261422820
Dates: received 25 August 2025; accepted 15 January 2026; published online 23 March 2026; in print March 2026
Type: Research article · Language: English
License: CC BY-NC
Identifiers: DOI 10.1177/03000605261422820 · PMID 41871206 · PMCID PMC13009808 · OpenAlex W7140132002
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: structural MRI / diffusion (modality), human (organism), other condition (population), methods / tools (subfield)
Methods: Connectivity, Machine learning
Keywords: You Only Look Once version 12 (YOLOv12n), brain tumor detection, Attention-based C2f with Frequency-domain Feed-Forward Network (A2C2f-DFFN) module, C2f with Token Statistics Self-Attention and Dynamic Tanh (C2TSSA-DYT) module, dynamic upsampling operator
MeSH: Algorithms*, Artificial Intelligence*, Brain Neoplasms*, Image Interpretation, Computer-Assisted*, Magnetic Resonance Imaging*, Detection Algorithms, Humans (* major topic)
Journal subjects: Observational Study
Topic: Brain Tumor Detection and Classification (Computer Vision and Pattern Recognition, Computer Science), according to OpenAlex
Citations: not cited yet (Europe PMC); 43 references in the paper

Abstract

Objective: To address challenges such as blurred boundaries and irregular shapes in brain tumor magnetic resonance imaging scans, we developed a lightweight detection framework to enhance automated diagnosis and meet real-time clinical requirements.

Methods: We proposed an improved You Only Look Once version 12 (YOLOv12n)-based model featuring three modules. First, the Attention-based C2f with Frequency-domain Feed-Forward Network (A2C2f-DFFN) module was incorporated into the backbone network; it combined an attention mechanism with a frequency-domain feedforward network to enhance global context modeling and detailed feature reconstruction. Second, the C2f with Token Statistics Self-Attention and Dynamic Tanh (C2TSSA-DYT) module was employed in the feature fusion neck; it utilized statistical self-attention and a dynamic Tanh activation function to improve robustness in complex backgrounds. Finally, the dynamic upsampling operator was adopted in the feature reconstruction stage; it dynamically generated sampling weights to effectively prevent boundary blurring and detail loss.

Results: On the Kaggle brain tumor dataset, our method achieved 93.2% precision, 88.4% recall, and 94.1% mean average precision at IoU threshold 0.5 (0.5), surpassing YOLOv12n and other lightweight models. It showed excellent performance in patients with glioma and pituitary tumor cases using only 6.0 Giga Floating-point Operations Per Second (GFLOPs) and 2.76 M parameters for efficient real-time inference.

Conclusion: The enhanced YOLOv12n framework proposed in this study achieved good balance between accuracy and efficiency in brain tumor detection tasks, demonstrating strong robustness, which makes it suitable for use in clinical computer-aided diagnosis systems.

Reproduced under the paper's license (CC BY-NC), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 11 matches between paragraphs and lines of code.

weilaibot/my_Brain_tumor

License: none: the authors keep all their rights
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: 680f06ba0c0f718d150b85643a9f5a149369e4a0, 15 November 2025
Languages: C/C++ (416), Python (311), CUDA (82), C++ (20), Shell (13), JavaScript (2)
Size: 1,763 files, 844 scripts
Software Heritage: not archived
Found in: “Data availability”
Holds: README, environment (shenduxuexilvcai_02/pyproject.toml, shenduxuexilvcai_02/requirements.txt, shenduxuexilvcai_02/docker/Dockerfile, shenduxuexilvcai_02/ultralytics/nn/extra_modules/DCNv4_op/setup.py, shenduxuexilvcai_02/ultralytics/nn/extra_modules/mamba/setup.py, shenduxuexilvcai_02/ultralytics/nn/extra_modules/ops_dcnv3/setup.py, shenduxuexilvcai_02/ultralytics/nn/extra_modules/rational_kat_cu/setup.py, shenduxuexilvcai_02/ultralytics/nn/extra_modules/selective_scan/setup.py, shenduxuexilvcai_02/ultralytics/nn/backbone/TransNeXt/swattention_extension/setup.py, shenduxuexilvcai_02/ultralytics/nn/extra_modules/cutlass/examples/19_large_depthwise_conv2d_torch_extension/setup.py), tests, documentation
Not found: license file, CITATION.cff, continuous integration
Tools: PyTorch (187 files), NumPy (84 files), OpenCV (28 files), Pillow (16 files), pandas (14 files), Matplotlib (13 files), SciPy (7 files), Hugging Face Transformers (5 files), seaborn (3 files), PyWavelets (2 files), scikit-learn (2 files), TensorFlow (2 files)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
845 files
At the source: github.com/weilaibot/my_Brain_tumor.git

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 844 scripts, each with its path and the digest of its content;
  • 11 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data availability

Publicly available datasets were used for this research. Code: https://github.com/weilaibot/my_Brain_tumor.git.

Reproduced under the paper's license (CC BY-NC), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 30 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 5 keywords, 7 MeSH terms, 18 references.

Cite

This paper

Zheng, R., & Cai, S. (2026). Brain tumor detection on magnetic resonance imaging scans using the artificial intelligence-based You Only Look Once algorithm. The Journal of international medical research, 54(3), 03000605261422820. https://doi.org/10.1177/03000605261422820

BibTeX

@article{zheng2026brain,
author = {Zheng, Ronghui and Cai, Shanshan},
title = {{Brain tumor detection on magnetic resonance imaging scans using the artificial intelligence-based You Only Look Once algorithm}},
journal = {The Journal of international medical research},
year = {2026},
month = mar,
volume = {54},
number = {3},
pages = {03000605261422820},
publisher = {SAGE Publications},
issn = {0300-0605},
doi = {10.1177/03000605261422820},
url = {https://doi.org/10.1177/03000605261422820},
pmid = {41871206},
pmcid = {PMC13009808}
}

RIS

TY - JOUR
AU - Zheng, Ronghui
AU - Cai, Shanshan
TI - Brain tumor detection on magnetic resonance imaging scans using the artificial intelligence-based You Only Look Once algorithm
T2 - The Journal of international medical research
J2 - J Int Med Res
PY - 2026
DA - 2026/03/23
VL - 54
IS - 3
SP - 03000605261422820
SN - 0300-0605
PB - SAGE Publications
DO - 10.1177/03000605261422820
UR - https://doi.org/10.1177/03000605261422820
LA - en
ER -

CSL-JSON

{
"id": "10.1177/03000605261422820",
"type": "article-journal",
"title": "Brain tumor detection on magnetic resonance imaging scans using the artificial intelligence-based You Only Look Once algorithm",
"container-title": "The Journal of international medical research",
"author": [
{
"family": "Zheng",
"given": "Ronghui"
},
{
"family": "Cai",
"given": "Shanshan"
}
],
"container-title-short": "J Int Med Res",
"volume": "54",
"issue": "3",
"page": "03000605261422820",
"DOI": "10.1177/03000605261422820",
"PMID": "41871206",
"PMCID": "PMC13009808",
"ISSN": "0300-0605",
"publisher": "SAGE Publications",
"URL": "https://doi.org/10.1177/03000605261422820",
"language": "en",
"issued": {
"date-parts": [
[
2026,
3,
23
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41598-024-66228-1 [code]
Image classification and reconstruction from low-density EEG.
Journal: Scientific reports
In common: PyWavelets, Hugging Face Transformers, OpenCV, 8 other tools, methods / tools, structural MRI / diffusion
[2] doi:10.1016/j.crmeth.2024.100964 [code]
Unifying community whole-brain imaging datasets enables robust neuron identification and reveals determinants of neuron position in C. elegans.
Journal: Cell reports methods
In common: PyWavelets, OpenCV, TensorFlow, 8 other tools, methods / tools
[3] doi:10.1038/s41467-026-72057-9 [code]
Sex-specific behavioral feedback modulates sensorimotor processing and drives flexible social behavior.
Journal: Nature communications
In common: PyWavelets, OpenCV, TensorFlow, 8 other tools
[4] doi:10.1038/s42003-026-10957-8 [code]
Brain defence by the extracellular matrix protein Cochlin.
Journal: Communications biology
In common: Hugging Face Transformers, OpenCV, TensorFlow, 8 other tools
[5] doi:10.1038/s41467-026-74358-5 [code]
Brain-inspired spatial intelligence for embodied agents.
Journal: Nature communications
In common: Hugging Face Transformers, OpenCV, TensorFlow, 8 other tools
[6] doi:10.1038/s41592-026-03057-2 [code]
CREsted: modeling genomic and synthetic cell-type-specific enhancers across tissues and species.
Journal: Nature methods
In common: PyWavelets, TensorFlow, Pillow, 7 other tools, methods / tools
[7] doi:10.1002/hbm.70469 [code]
VarCoNet: A Variability-Aware Self-Supervised Framework for Functional Connectome Extraction From Resting-State fMRI.
Journal: Human brain mapping
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools, methods / tools
[8] doi:10.1093/braincomms/fcag253 [code]
Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study.
Journal: Brain communications
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools, structural MRI / diffusion
[9] doi:10.1016/j.patter.2026.101538 [code]
A multi-modal foundation model for brain disease diagnosis and medical imaging.
Journal: Patterns (New York, N.Y.)
In common: Hugging Face Transformers, OpenCV, TensorFlow, 7 other tools
[10] doi:10.1038/s41467-026-76837-1 [code]
Drug screen and machine learning predict neuroprotective agents in a preclinical human model of childhood dementia.
Journal: Nature communications
In common: Hugging Face Transformers, OpenCV, Pillow, 7 other tools, other condition

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1177/03000605261422820.svg)](https://openscicode.org/badge/doi/10.1177/03000605261422820)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1177/03000605261422820.svg
   :target: https://openscicode.org/badge/doi/10.1177/03000605261422820
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1177/03000605261422820"><img src="https://openscicode.org/badge/doi/10.1177/03000605261422820.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.