OSCR

Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion.

Code ↔ Paper

3 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

3 blocks of code here have a permanent identifier (oscr:b:…), listed with the matches below: citable for good, with a page of its own. Select a match and press Cite to copy it. What a block identifier is.

The 3 matches
  1. [1] § Results › Tera-MIND accurately generated tera-scale mouse brain(s) by spatial gene expression ↔ utils/metrics.py, lines 201 to 215 · score 0.59 · peak signal, noise ratio, metrics, PSNR · block oscr:b:ZVJS2X72WA9KX-F · Report a mismatch
  2. [2] § Results › Tera-MIND achieved reproducible and robust results on three tera-scale mouse brains ↔ utils/vis_mba.py, lines 278 to 340 · score 0.56 · Nr4a2, Slc17a6, Slc17a7, Pathway, DOPA, GLUT · block oscr:b:A4MZBPC38HRAK-L · Report a mismatch
  3. [3] § Results › Tera-MIND achieved reproducible and robust results on three tera-scale mouse brains ↔ utils/__init__.py, lines 59 to 95 · score 0.52 · Nr4a2, Slc17a6, Slc17a7, DOPA, GLUT · block oscr:b:TXJ625PXKVAS2-E · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 557 lines · 18 KB · MIT · 1 match

  1. import torch
  2. import pickle
  3. import random
  4. import numpy as np
  5. import torch.nn.functional as F
  6. import matplotlib.pyplot as plt
  7. from cellpose import plot as cplt
  8. import warnings
  9. from torch.linalg import eigvals
  10. def is_valid(pnm, gnm):
  11. for p in pnm:
  12. if p not in gnm:
  13. return False
  14. return True
  15. def dt_region(hst, wst, hnm, wnm,
  16. size=256, is_gt=True):
  17. pad = size // 2
  18. dt_lst = []
  19. for ph in range(hnm):
  20. hsz = ph * size
  21. for pw in range(wnm):
  22. wsz = pw * size
  23. pnm = [hst + hsz, hst + size + hsz,
  24. wst + wsz, wst + size + wsz]
  25. if is_gt:
  26. pnm += [hst - pad + hsz, hst + size + pad + hsz,
  27. wst - pad + wsz, wst + size + pad + wsz]
  28. pnm = '_'.join([str(p) for p in pnm])
  29. dt_lst.append(pnm)
  30. return dt_lst
  31. def dt_sublst(gnm, hst=256, wst=256, hnm=16, wnm=16,
  32. size=256, step=1, is_gt=True):
  33. # for 1024 x 1024, hst=wst=512,
  34. # hnm=412, wnm=284, step = 4
  35. dt_lst= []
  36. # Here, we assume no boundary issue
  37. for pw in range(0, wnm, step):
  38. wsz = pw * size
  39. for ph in range(0, hnm, step):
  40. hsz = ph * size
  41. dt_reg = dt_region(hst+hsz, wst+wsz,
  42. step, step, is_gt=is_gt)
  43. if is_valid(dt_reg, gnm):
  44. dt_lst.append(dt_reg)
  45. return dt_lst
  46. def _d_novel(sigma1, sigma2):
  47. r"""
  48. The core and more efficient impl of d_FID
  49. Args:
  50. sigma1: Covariance of one image collection
  51. sigma2: Covariance of compared image collection
  52. """
  53. eigval = eigvals(sigma1 @ sigma2)
  54. eigval = eigval.real
  55. eigval[eigval < 0] = 0
  56. return 2 * eigval.sqrt().sum(-1)
  57. def calc_d_fid(mu1, mu2, sigma1, sigma2):
  58. r"""
  59. The Function of d_FID calc
  60. Args:
  61. mu1: Mean of one image feat collection
  62. mu2: Mean of compared image feat collection
  63. sigma1: Covariance of one image feat collection
  64. sigma2: Covariance of compared image feat collection
  65. """
  66. mu1 = torch.atleast_1d(mu1)
  67. mu2 = torch.atleast_1d(mu2)
  68. sigma1 = torch.atleast_2d(sigma1)
  69. sigma2 = torch.atleast_2d(sigma2)
  70. assert mu1.shape == mu2.shape, \
  71. 'Training and test mean vectors have different lengths'
  72. assert sigma1.shape == sigma2.shape, \
  73. 'Training and test covariances have different dimensions'
  74. diff = mu1 - mu2
  75. fid_easy = diff.dot(diff) + torch.trace(sigma1) + torch.trace(sigma2)
  76. fid_hard = _d_novel(sigma1, sigma2)
  77. fid = fid_easy - fid_hard
  78. return fid
  79. def calc_d_fid3(mu1, mu2, sigma1, sigma2):
  80. mu1 = torch.atleast_2d(mu1)
  81. mu2 = torch.atleast_2d(mu2)
  82. sigma1 = torch.atleast_3d(sigma1)
  83. sigma2 = torch.atleast_3d(sigma2)
  84. assert mu1.shape == mu2.shape, \
  85. 'Training and test mean vectors have different lengths'
  86. assert sigma1.shape == sigma2.shape, \
  87. 'Training and test covariances have different dimensions'
  88. dif = mu1 - mu2
  89. fid_easy = (dif ** 2).sum(-1) + torch.vmap(torch.trace)(sigma1) + torch.vmap(torch.trace)(sigma2)
  90. fid_hard = _d_novel(sigma1, sigma2)
  91. fid = fid_easy - fid_hard
  92. return fid
  93. def calc_mean_var(mu, scm, tot):
  94. mu = mu / tot[..., None]
  95. scm = scm / tot[..., None, None]
  96. sigma = scm - mu.unsqueeze(-1) @ mu.unsqueeze(-2)
  97. return mu, sigma
  98. def calc_slc_all_fid(m_r, s_r, tot_r,
  99. m_g, s_g, tot_g, is_str=False):
  100. m_r0, s_r0 = calc_mean_var(m_r, s_r, tot_r)
  101. m_g0, s_g0 = calc_mean_var(m_g, s_g, tot_g)
  102. fid0 = calc_d_fid3(m_r0, m_g0, s_r0, s_g0)
  103. m_r1, s_r1 = calc_mean_var(m_r.sum(0)[None], s_r.sum(0)[None], tot_r.sum(0)[None])
  104. m_g1, s_g1 = calc_mean_var(m_g.sum(0)[None], s_g.sum(0)[None], tot_g.sum(0)[None])
  105. fid1 = calc_d_fid3(m_r1, m_g1, s_r1, s_g1)
  106. fid = torch.cat((fid0, fid1)).cpu().numpy()
  107. if is_str:
  108. return [f'{fd:.2f}' for fd in fid]
  109. return fid
  110. def calc_mean_std_msk(stat, dims):
  111. msk = stat!=0
  112. avg = (stat*msk).sum(dims)/msk.sum(dims)
  113. # Here, we always assume (batch, z) for stat
  114. var = (((stat - avg)**2)*msk).sum(dims)/msk.sum(dims)
  115. return avg, var.sqrt()
  116. def calc_slc_all_1d(stat, is_str=False):
  117. avg0, std0 = calc_mean_std_msk(stat, 0)
  118. avg1, std1 = calc_mean_std_msk(stat, (0, 1))
  119. avg = torch.cat((avg0, avg1[None]))
  120. std = torch.cat((std0, std1[None]))
  121. if is_str:
  122. return [f'{a:.2f} {s:.2f}' for a, s in zip(avg, std)]
  123. return avg, std
  124. def calc_cellpose(img, mod, met,
  125. pth, roi, debug):
  126. im = img.astype('float') / 255.
  127. im_lst = np.split(im[:, 0], im.shape[0], 0)
  128. im_lst = [(v, i[0]) for v, i in enumerate(im_lst) if (img[v, 0] != 0).any()]
  129. if not im_lst:
  130. print(f'{roi} all black, ignore.')
  131. return
  132. vl_lst, mk_lst = zip(*im_lst)
  133. masks, flows, styles, diams = mod.eval(
  134. list(mk_lst), diameter=None,
  135. normalize=False, channels=[0, 0])
  136. expr = im.mean((-1, -2))
  137. out = [torch.zeros(3, im.shape[0]),
  138. # Estimate the maximum num of cells
  139. torch.zeros(512, im.shape[0]),
  140. vl_lst]
  141. for mid, msk in zip(vl_lst, masks):
  142. cnt = np.unique(msk, return_counts=True)[1]
  143. cnt = cnt[1:]
  144. out[0][0, mid] = len(cnt)
  145. out[0][1:, mid] = torch.FloatTensor(expr[mid])
  146. out[1][:len(cnt), mid] = torch.FloatTensor(cnt)
  147. mlen = int(out[0][0].max())
  148. assert mlen <= out[1].shape[0]
  149. out[1] = out[1][:mlen]
  150. met['nstat'].append(out[0])
  151. met['narea'].append(out[1])
  152. met['valid'].append(out[2])
  153. if debug:
  154. print(roi, vl_lst, expr.shape, out[1].shape)
  155. for mid in range(im.shape[0]):
  156. if mid not in vl_lst:
  157. if (expr[mid] != 0).any():
  158. print(mid, expr[mid])
  159. _s = random.choice(range(len(vl_lst)))
  160. fig = plt.figure(figsize=(12, 5))
  161. cplt.show_segmentation(fig, img[vl_lst[_s], 0], masks[_s],
  162. flows[_s][0], channels=[0, 0])
  163. plt.tight_layout()
  164. plt.savefig(str(pth / f'{roi}_{vl_lst[_s]}.png'),
  165. bbox_inches='tight', dpi=200)
  166. plt.close()
  167. class PSNR(torch.nn.Module):
  168. """Peak Signal to Noise Ratio
  169. img1 and img2 have range [0, 255]"""
  170. def __init__(self, mval=255.):
  171. super(PSNR, self).__init__()
  172. self.mval = mval
  173. def forward(self, img1, img2):
  174. if len(img1.shape) == 3:
  175. dim = [1, 2]
  176. elif len(img1.shape) == 4:
  177. dim = [1, 2, 3]
  178. mse = torch.mean((img1 - img2) ** 2, dim=dim)
  179. return 20 * torch.log10(self.mval / torch.sqrt(mse))
  180. def _fspecial_gauss_1d(size, sigma):
  181. r"""Create 1-D gauss kernel
  182. Args:
  183. size (int): the size of gauss kernel
  184. sigma (float): sigma of normal distribution
  185. Returns:
  186. torch.Tensor: 1D kernel (1 x 1 x size)
  187. """
  188. coords = torch.arange(size, dtype=torch.float)
  189. coords -= size // 2
  190. g = torch.exp(-(coords ** 2) / (2 * sigma ** 2))
  191. g /= g.sum()
  192. return g.unsqueeze(0).unsqueeze(0)
  193. def gaussian_filter(input, win):
  194. r""" Blur input with 1-D kernel
  195. Args:
  196. input (torch.Tensor): a batch of tensors to be blurred
  197. window (torch.Tensor): 1-D gauss kernel
  198. Returns:
  199. torch.Tensor: blurred tensors
  200. """
  201. assert all([ws == 1 for ws in win.shape[1:-1]]), win.shape
  202. if len(input.shape) == 4:
  203. conv = F.conv2d
  204. elif len(input.shape) == 5:
  205. conv = F.conv3d
  206. else:
  207. raise NotImplementedError(input.shape)
  208. C = input.shape[1]
  209. out = input
  210. for i, s in enumerate(input.shape[2:]):
  211. if s >= win.shape[-1]:
  212. out = conv(out, weight=win.transpose(2 + i, -1), stride=1, padding=0, groups=C)
  213. else:
  214. warnings.warn(
  215. f"Skipping Gaussian Smoothing at dimension 2+{i} for input: {input.shape} and win size: {win.shape[-1]}"
  216. )
  217. return out
  218. def _ssim(X, Y, data_range, win, size_average=True, K=(0.01, 0.03)):
  219. r""" Calculate ssim index for X and Y
  220. Args:
  221. X (torch.Tensor): images
  222. Y (torch.Tensor): images
  223. win (torch.Tensor): 1-D gauss kernel
  224. data_range (float or int, optional): value range of input images. (usually 1.0 or 255)
  225. size_average (bool, optional): if size_average=True, ssim of all images will be averaged as a scalar
  226. Returns:
  227. torch.Tensor: ssim results.
  228. """
  229. K1, K2 = K
  230. # batch, channel, [depth,] height, width = X.shape
  231. compensation = 1.0
  232. C1 = (K1 * data_range) ** 2
  233. C2 = (K2 * data_range) ** 2
  234. win = win.to(X.device, dtype=X.dtype)
  235. mu1 = gaussian_filter(X, win)
  236. mu2 = gaussian_filter(Y, win)
  237. mu1_sq = mu1.pow(2)
  238. mu2_sq = mu2.pow(2)
  239. mu1_mu2 = mu1 * mu2
  240. sigma1_sq = compensation * (gaussian_filter(X * X, win) - mu1_sq)
  241. sigma2_sq = compensation * (gaussian_filter(Y * Y, win) - mu2_sq)
  242. sigma12 = compensation * (gaussian_filter(X * Y, win) - mu1_mu2)
  243. cs_map = (2 * sigma12 + C2) / (sigma1_sq + sigma2_sq + C2) # set alpha=beta=gamma=1
  244. ssim_map = ((2 * mu1_mu2 + C1) / (mu1_sq + mu2_sq + C1)) * cs_map
  245. ssim_per_channel = torch.flatten(ssim_map, 2).mean(-1)
  246. cs = torch.flatten(cs_map, 2).mean(-1)
  247. return ssim_per_channel, cs
  248. def ssim(
  249. X,
  250. Y,
  251. data_range=255,
  252. size_average=True,
  253. win_size=11,
  254. win_sigma=1.5,
  255. win=None,
  256. K=(0.01, 0.03),
  257. nonnegative_ssim=False,
  258. ):
  259. r""" interface of ssim
  260. Args:
  261. X (torch.Tensor): a batch of images, (N,C,H,W)
  262. Y (torch.Tensor): a batch of images, (N,C,H,W)
  263. data_range (float or int, optional): value range of input images. (usually 1.0 or 255)
  264. size_average (bool, optional): if size_average=True, ssim of all images will be averaged as a scalar
  265. win_size: (int, optional): the size of gauss kernel
  266. win_sigma: (float, optional): sigma of normal distribution
  267. win (torch.Tensor, optional): 1-D gauss kernel. if None, a new kernel will be created according to win_size and win_sigma
  268. K (list or tuple, optional): scalar constants (K1, K2). Try a larger K2 constant (e.g. 0.4) if you get a negative or NaN results.
  269. nonnegative_ssim (bool, optional): force the ssim response to be nonnegative with relu
  270. Returns:
  271. torch.Tensor: ssim results
  272. """
  273. if not X.shape == Y.shape:
  274. raise ValueError(f"Input images should have the same dimensions, but got {X.shape} and {Y.shape}.")
  275. for d in range(len(X.shape) - 1, 1, -1):
  276. X = X.squeeze(dim=d)
  277. Y = Y.squeeze(dim=d)
  278. if len(X.shape) not in (4, 5):
  279. raise ValueError(f"Input images should be 4-d or 5-d tensors, but got {X.shape}")
  280. if not X.type() == Y.type():
  281. raise ValueError(f"Input images should have the same dtype, but got {X.type()} and {Y.type()}.")
  282. if win is not None: # set win_size
  283. win_size = win.shape[-1]
  284. if not (win_size % 2 == 1):
  285. raise ValueError("Window size should be odd.")
  286. if win is None:
  287. win = _fspecial_gauss_1d(win_size, win_sigma)
  288. win = win.repeat([X.shape[1]] + [1] * (len(X.shape) - 1))
  289. ssim_per_channel, cs = _ssim(X, Y, data_range=data_range, win=win, size_average=False, K=K)
  290. if nonnegative_ssim:
  291. ssim_per_channel = torch.relu(ssim_per_channel)
  292. if size_average:
  293. return ssim_per_channel.mean()
  294. else:
  295. return ssim_per_channel.mean(1)
  296. def ms_ssim(
  297. X, Y, data_range=255, size_average=True, win_size=11, win_sigma=1.5, win=None, weights=None, K=(0.01, 0.03)
  298. ):
  299. r""" interface of ms-ssim
  300. Args:
  301. X (torch.Tensor): a batch of images, (N,C,[T,]H,W)
  302. Y (torch.Tensor): a batch of images, (N,C,[T,]H,W)
  303. data_range (float or int, optional): value range of input images. (usually 1.0 or 255)
  304. size_average (bool, optional): if size_average=True, ssim of all images will be averaged as a scalar
  305. win_size: (int, optional): the size of gauss kernel
  306. win_sigma: (float, optional): sigma of normal distribution
  307. win (torch.Tensor, optional): 1-D gauss kernel. if None, a new kernel will be created according to win_size and win_sigma
  308. weights (list, optional): weights for different levels
  309. K (list or tuple, optional): scalar constants (K1, K2). Try a larger K2 constant (e.g. 0.4) if you get a negative or NaN results.
  310. Returns:
  311. torch.Tensor: ms-ssim results
  312. """
  313. if not X.shape == Y.shape:
  314. raise ValueError(f"Input images should have the same dimensions, but got {X.shape} and {Y.shape}.")
  315. for d in range(len(X.shape) - 1, 1, -1):
  316. X = X.squeeze(dim=d)
  317. Y = Y.squeeze(dim=d)
  318. if not X.type() == Y.type():
  319. raise ValueError(f"Input images should have the same dtype, but got {X.type()} and {Y.type()}.")
  320. if len(X.shape) == 4:
  321. avg_pool = F.avg_pool2d
  322. elif len(X.shape) == 5:
  323. avg_pool = F.avg_pool3d
  324. else:
  325. raise ValueError(f"Input images should be 4-d or 5-d tensors, but got {X.shape}")
  326. if win is not None: # set win_size
  327. win_size = win.shape[-1]
  328. if not (win_size % 2 == 1):
  329. raise ValueError("Window size should be odd.")
  330. smaller_side = min(X.shape[-2:])
  331. assert smaller_side > (win_size - 1) * (
  332. 2 ** 4
  333. ), "Image size should be larger than %d due to the 4 downsamplings in ms-ssim" % ((win_size - 1) * (2 ** 4))
  334. if weights is None:
  335. weights = [0.0448, 0.2856, 0.3001, 0.2363, 0.1333]
  336. weights = X.new_tensor(weights)
  337. if win is None:
  338. win = _fspecial_gauss_1d(win_size, win_sigma)
  339. win = win.repeat([X.shape[1]] + [1] * (len(X.shape) - 1))
  340. levels = weights.shape[0]
  341. mcs = []
  342. for i in range(levels):
  343. ssim_per_channel, cs = _ssim(X, Y, win=win, data_range=data_range, size_average=False, K=K)
  344. if i < levels - 1:
  345. mcs.append(torch.relu(cs))
  346. padding = [s % 2 for s in X.shape[2:]]
  347. X = avg_pool(X, kernel_size=2, padding=padding)
  348. Y = avg_pool(Y, kernel_size=2, padding=padding)
  349. ssim_per_channel = torch.relu(ssim_per_channel) # (batch, channel)
  350. mcs_and_ssim = torch.stack(mcs + [ssim_per_channel], dim=0) # (level, batch, channel)
  351. ms_ssim_val = torch.prod(mcs_and_ssim ** weights.view(-1, 1, 1), dim=0)
  352. if size_average:
  353. return ms_ssim_val.mean()
  354. else:
  355. return ms_ssim_val.mean(1)
  356. class SSIM(torch.nn.Module):
  357. def __init__(
  358. self,
  359. data_range=255,
  360. size_average=True,
  361. win_size=11,
  362. win_sigma=1.5,
  363. channel=3,
  364. spatial_dims=2,
  365. K=(0.01, 0.03),
  366. nonnegative_ssim=False,
  367. unsqueeze=False
  368. ):
  369. r""" class for ssim
  370. Args:
  371. data_range (float or int, optional): value range of input images. (usually 1.0 or 255)
  372. size_average (bool, optional): if size_average=True, ssim of all images will be averaged as a scalar
  373. win_size: (int, optional): the size of gauss kernel
  374. win_sigma: (float, optional): sigma of normal distribution
  375. channel (int, optional): input channels (default: 3)
  376. K (list or tuple, optional): scalar constants (K1, K2). Try a larger K2 constant (e.g. 0.4) if you get a negative or NaN results.
  377. nonnegative_ssim (bool, optional): force the ssim response to be nonnegative with relu.
  378. """
  379. super(SSIM, self).__init__()
  380. self.win_size = win_size
  381. self.win = _fspecial_gauss_1d(win_size, win_sigma).repeat([channel, 1] + [1] * spatial_dims)
  382. self.size_average = size_average
  383. self.data_range = data_range
  384. self.K = K
  385. self.nonnegative_ssim = nonnegative_ssim
  386. self.unsqueeze = unsqueeze
  387. def forward(self, X, Y):
  388. if self.unsqueeze:
  389. # this suggests that the input
  390. # has only 3 channels [N, H, W]
  391. # then unsqueeze the C channel
  392. assert len(X.shape) == len(Y.shape) == 3
  393. X = X.unsqueeze(1)
  394. Y = Y.unsqueeze(1)
  395. return ssim(
  396. X,
  397. Y,
  398. data_range=self.data_range,
  399. size_average=self.size_average,
  400. win=self.win,
  401. K=self.K,
  402. nonnegative_ssim=self.nonnegative_ssim,
  403. )
  404. class MS_SSIM(torch.nn.Module):
  405. def __init__(
  406. self,
  407. data_range=255,
  408. size_average=True,
  409. win_size=11,
  410. win_sigma=1.5,
  411. channel=3,
  412. spatial_dims=2,
  413. weights=None,
  414. K=(0.01, 0.03),
  415. unsqueeze=False
  416. ):
  417. r""" class for ms-ssim
  418. Args:
  419. data_range (float or int, optional): value range of input images. (usually 1.0 or 255)
  420. size_average (bool, optional): if size_average=True, ssim of all images will be averaged as a scalar
  421. win_size: (int, optional): the size of gauss kernel
  422. win_sigma: (float, optional): sigma of normal distribution
  423. channel (int, optional): input channels (default: 3)
  424. weights (list, optional): weights for different levels
  425. K (list or tuple, optional): scalar constants (K1, K2). Try a larger K2 constant (e.g. 0.4) if you get a negative or NaN results.
  426. """
  427. super(MS_SSIM, self).__init__()
  428. self.win_size = win_size
  429. self.win = _fspecial_gauss_1d(win_size, win_sigma).repeat([channel, 1] + [1] * spatial_dims)
  430. self.win_gray = _fspecial_gauss_1d(win_size, win_sigma).repeat([1, 1] + [1] * spatial_dims)
  431. self.size_average = size_average
  432. self.data_range = data_range
  433. self.weights = weights
  434. self.K = K
  435. self.unsqueeze = unsqueeze
  436. def forward(self, X, Y):
  437. is_gray = False
  438. if len(X.shape) == 3:
  439. X, Y = X[:, None], Y[:, None]
  440. is_gray = True
  441. return ms_ssim(
  442. X,
  443. Y,
  444. data_range=self.data_range,
  445. size_average=self.size_average,
  446. win=self.win_gray if is_gray else self.win,
  447. weights=self.weights,
  448. K=self.K,
  449. )
  450. if __name__ == '__main__':
  451. import torch
  452. # mu = torch.rand(3, 50)
  453. # scm = torch.rand(3, 50, 50)
  454. # tot = torch.rand(3)
  455. # a = mu.clone()
  456. # m0, s0 = calc_mean_var(mu, scm, tot)
  457. # print((a == mu).all())
  458. # ssim_ = SSIM(channel=2, size_average=False)
  459. # a = torch.rand(23, 2, 125,125)
  460. # b = torch.rand(23, 2, 125,125)
  461. # c = ssim_(a, b)
  462. # print(c.shape)

metrics.py at commit 083747d, under MIT · at the source

Overview

Authors: Jiqing Wu1, Ingrid Berg2, Yawei Li3,4, Ender Konukoglu3, Viktor H Koelzer1,5
ORCID iDs: Jiqing Wu
  1. Department of Biomedical Engineering, University of Basel, Basel, Switzerland
  2. Department of Pathology and Molecular Pathology, University Hospital, University of Zurich, Zurich, Switzerland
  3. Computer Vision Lab, ETH Zurich, Zurich, Switzerland
  4. Integrated System Laboratory, ETH Zurich, Zurich, Switzerland
  5. Institute of Medical Genetics and Pathology, University Hospital Basel, Basel, Switzerland
Institutions: University of Basel (Switzerland); University of Zurich (Switzerland); University Hospital Zurich (Switzerland); ETH Zurich (Switzerland); Integrated Systems Laboratory (Switzerland); Computer Vision Laboratory (Switzerland); University Hospital of Basel (Switzerland)
Journal: iScience, volume 29, issue 7, article 116355
Dates: received 10 October 2025; accepted 6 April 2026; published online 8 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1016/j.isci.2026.116355 · PMID 42491506 · PMCID PMC13378011 · OpenAlex W4415338870
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: genetics / omics (modality), mouse (organism), cellular / molecular (subfield)
Methods: Machine learning
Keywords: Neuroscience, Biocomputational method, Transcriptomics
Topic: RNA Research and Splicing (Molecular Biology, Biochemistry, Genetics and Molecular Biology), according to OpenAlex
Funding: Universitätsspital Basel
Citations: not cited yet (Europe PMC); 52 references in the paper

Abstract

Holistic 3D modeling of molecularly defined brain structures is crucial for understanding complex brain functions. Using emerging tissue profiling technologies, researchers charted comprehensive atlases of mammalian brain with sub-cellular resolution and spatially resolved transcriptomic data. However, these tera-scale volumetric atlases pose computational challenges for modeling intricate brain structures within the native spatial context. We propose Tera-MIND, a novel generative framework capable of simulating Tera-scale mouse brains in 3D using a patch-based and boundary-aware diffusion model. Taking spatial gene expression as conditional input, we generate virtual mouse brains with comprehensive cellular morphological detail at teravoxel scale. Through the lens of 3D gene-gene self-attention, we identify spatial molecular interactions for key transcriptomic pathways, including glutamatergic and dopaminergic neuronal systems. Lastly, we showcase the translational applicability of Tera-MIND on previously unseen human brain samples. Tera-MIND offers an efficient generative modeling of whole virtual organisms, paving the way for integrative applications in biomedical research.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 3 matches between paragraphs and lines of code.

Zenodo 14826874

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Size: 0 files, 0 scripts
Software Heritage: not checked
Found in: “Data and code availability”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
  • 27 September 2026: the link answers (HTTP 200)
At the source:

CTPLab/Tera-MIND

License: MIT
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 083747d0ddc3d17ccbb31c5556b9f319d375852f, 24 June 2025
Languages: Python (32), Shell (7)
Size: 52 files, 39 scripts
Software Heritage: not archived
Found in: “Data and code availability”
Holds: README, license file, environment (environment.yml)
Not found: CITATION.cff, tests, continuous integration, documentation
Tools: PyTorch (23 files), NumPy (15 files), pandas (7 files), Pillow (6 files), Matplotlib (4 files), Cellpose (3 files), seaborn (3 files), OpenCV (2 files), PyTorch Lightning (1 file), SciPy (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
41 files

musikisomorphie/tera-mind

License: none: the authors keep all their rights
State: the link is dead, verified on 27 September 2026
Evidence: found in the paper
Software Heritage: not archived
Found in: the Zenodo archive record
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Availability: 1 check, the latest on 27 September 2026: the link is dead
  • 27 September 2026: the link is dead

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 3 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 39 scripts, each with its path and the digest of its content;
  • 3 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data and code availability

The original mouse brain data were deposited at brain image library under the database identifier ace-den-fix. The processed mouse brain data were deposited at brain image library under the database identifier ace-lot-now and are publicly available as of the date of publication. The human brain data are available via https://www.10xgenomics.com/datasets/xenium-human-brain-preview-data-1-standard.

All original code has been deposited at Zenodo under the https://doi.org/10.5281/zenodo.14826874 or https://github.com/CTPLab/Tera-MIND.

Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

  • Authors: added Jiqing Wu (0000-0002-6898-8698); removed Jiqing Wu

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 5 authors, 3 keywords, 1 funder, 36 references.

Cite

This paper

Wu, J., Berg, I., Li, Y., Konukoglu, E., & Koelzer, V. H. (2026). Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion. iScience, 29(7), 116355. https://doi.org/10.1016/j.isci.2026.116355

BibTeX

@article{wu2026tera,
author = {Wu, Jiqing and Berg, Ingrid and Li, Yawei and Konukoglu, Ender and Koelzer, Viktor H},
title = {{Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion}},
journal = {iScience},
year = {2026},
month = jul,
volume = {29},
number = {7},
pages = {116355},
publisher = {Elsevier},
issn = {2589-0042},
doi = {10.1016/j.isci.2026.116355},
url = {https://doi.org/10.1016/j.isci.2026.116355},
pmid = {42491506},
pmcid = {PMC13378011}
}

RIS

TY - JOUR
AU - Wu, Jiqing
AU - Berg, Ingrid
AU - Li, Yawei
AU - Konukoglu, Ender
AU - Koelzer, Viktor H
TI - Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion
T2 - iScience
J2 - iScience
PY - 2026
DA - 2026/07/08
VL - 29
IS - 7
SP - 116355
SN - 2589-0042
PB - Elsevier
DO - 10.1016/j.isci.2026.116355
UR - https://doi.org/10.1016/j.isci.2026.116355
LA - en
ER -

CSL-JSON

{
"id": "10.1016/j.isci.2026.116355",
"type": "article-journal",
"title": "Tera-MIND: Tera-scale mouse brain simulation via spatial mRNA-guided diffusion",
"container-title": "iScience",
"author": [
{
"family": "Wu",
"given": "Jiqing"
},
{
"family": "Berg",
"given": "Ingrid"
},
{
"family": "Li",
"given": "Yawei"
},
{
"family": "Konukoglu",
"given": "Ender"
},
{
"family": "Koelzer",
"given": "Viktor H"
}
],
"container-title-short": "iScience",
"volume": "29",
"issue": "7",
"page": "116355",
"DOI": "10.1016/j.isci.2026.116355",
"PMID": "42491506",
"PMCID": "PMC13378011",
"ISSN": "2589-0042",
"publisher": "Elsevier",
"URL": "https://doi.org/10.1016/j.isci.2026.116355",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
8
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41598-026-46455-4 [code]
Generative cerebral vasculature visualization using spatial transcriptomic data.
Journal: Scientific reports
In common: Cellpose, PyTorch Lightning, OpenCV, 7 other tools, genetics / omics, mouse, cellular / molecular, 5 references
[2] doi:10.1016/j.xgen.2024.100739 [code]
CSI-GEP: A GPU-based unsupervised machine learning approach for recovering gene expression programs in atlas-scale single-cell RNA-seq data.
Journal: Cell genomics
In common: PyTorch Lightning, Pillow, PyTorch, 5 other tools, genetics / omics, mouse, cellular / molecular, 4 references
[3] doi:10.7554/elife.90029 [code]
Three-dimensional single-cell transcriptome imaging of thick tissues.
Journal: eLife
In common: Cellpose, OpenCV, Pillow, 5 other tools, genetics / omics, mouse, cellular / molecular, 3 references
[4] doi:10.1126/science.adq2084 [code]
Deep-tissue transcriptomics and subcellular imaging at high spatial resolution.
Journal: Science (New York, N.Y.)
In common: Cellpose, OpenCV, PyTorch, 5 other tools, genetics / omics, mouse, 3 references
[5] doi:10.1038/s41587-022-01455-3 [code]
Scalable in situ single-cell profiling by electrophoretic capture of mRNA using EEL FISH.
Journal: Nature biotechnology
In common: Cellpose, PyTorch Lightning, OpenCV, 5 other tools, genetics / omics, mouse, cellular / molecular, 2 references
[6] doi:10.1038/s41467-026-71759-4 [code]
CellNiche represents cellular microenvironments in atlas-scale spatial omics data with contrastive learning.
Journal: Nature communications
In common: PyTorch, seaborn, pandas, 3 other tools, mouse, cellular / molecular, 6 references
[7] doi:10.1371/journal.pcbi.1014571 [code]
SynAPSeg: A novel dataset and image analysis framework for deep learning-based synapse detection and quantification.
Journal: PLoS computational biology
In common: Cellpose, OpenCV, Pillow, 6 other tools, 2 references
[8] doi:10.1038/s41588-025-02120-6 [code]
Quantitative characterization of cell niches in spatially resolved omics data.
Journal: Nature genetics
In common: PyTorch, seaborn, pandas, 3 other tools, genetics / omics, mouse, 6 references
[9] doi:10.1038/s41467-025-64259-4 [code]
Data-driven fine-grained region discovery in the mouse brain with transformers.
Journal: Nature communications
In common: PyTorch Lightning, PyTorch, seaborn, 4 other tools, genetics / omics, mouse, 4 references
[10] doi:10.1017/s2633903x24000114 [code]
Bridging the gap: Integrating cutting-edge techniques into biological imaging with deepImageJ.
Journal: Biological imaging
In common: Cellpose, PyTorch Lightning, OpenCV, 6 other tools, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355.svg)](https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355.svg
   :target: https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355"><img src="https://openscicode.org/badge/doi/10.1016/j.isci.2026.116355.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.