OSCR

Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions Through Logical Reasoning.

Code ↔ Paper

8 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 8 matches · 1 of them tie a paragraph to a whole file, not to given lines: a weak match, whose lines are not tinted
  1. [1] § Appendix 3: Experimental Setup ↔ src/npc-models/header.py, lines 1–67 · score 0.76 · concept loss weight, weight decay, momentum, embedding, validation, seeds
  2. [2] § Preliminaries ↔ src/npc-models/pc.py, lines 113–144 · score 0.67 · weighted sum, weight updating, root node, sum node, children, joint
  3. [3] § Neural Probabilistic Circuits › Three-Stage Training Algorithm › Circuit Construction ↔ src/npc-models/pc.py, lines 388–404 · score 0.66 · root node, leaf nodes, product node, sum node, depth
  4. [4] § Appendix 3: Experimental Setup ↔ src/npc-models/train_npc.py, lines 199–337 · score 0.64 · weight decay, SGD, plateaus, momentum, validation, seeds
  5. [5] § Preliminaries ↔ src/npc-models/train_npc.py, lines 199–337 · score 0.62 · leaf nodes, product node, sum node, Probabilistic circuits, transformed, loss
  6. [6] § Neural Probabilistic Circuits › Three-Stage Training Algorithm › Circuit Construction ↔ src/npc-models/train_pc.py, lines 61–151 · score 0.60 · leaf nodes, product nodes, sum nodes, likelihood, circuit, training
  7. [7] § Experiments › Ablation Studies › Impact of Interventions ↔ scripts/npc-models/train_npc.bash, the whole file · a weak match · score 0.59 · trained NPC models, joint optimization, CelebA, MNIST, GTSRB, AwA2
  8. [8] § Preliminaries ↔ src/npc-models/train_pc.py, lines 61–151 · score 0.56 · leaf node, product node, sum node, marginal, circuits, joint

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 829 lines · 32 KB · CC-BY-NC-SA-4.0 · 2 matches

  1. """
  2. @file pc.py
  3. @author Simon Yu
  4. @date 02/07/2024
  5. @brief PC classes.
  6. """
  8. import abc
  9. import itertools
  10. import logger
  11. import numpy
  12. import os
  13. import torch
  14. import tqdm
  16. class PCLearningRateScheduler:
  17. def __init__(self, optimizer, factor = 0.8, patience = 2, threshold = 1e-4, cooldown = 2, min_learning_rate = 1e-6):
  18. self.cooldown = cooldown
  19. self.cooldown_counter = 0
  20. self.factor = factor
  21. self.metrics = None
  22. self.min_learning_rate = min_learning_rate
  23. self.optimizer = optimizer
  24. self.patience = patience
  25. self.patience_counter = 0
  26. self.threshold = threshold
  28. return
  30. @abc.abstractmethod
  31. def step(self, metrics):
  32. pass
  34. class LikelihoodPCLearningRateScheduler(PCLearningRateScheduler):
  35. def __init__(self, optimizer, factor = 0.8):
  36. super().__init__(optimizer, factor)
  38. return
  40. def step(self, metrics):
  41. if self.metrics is not None:
  42. if metrics < self.metrics:
  43. self.optimizer.learning_rate *= self.factor
  44. logger.log_info("Reducing PC learning rate to " + "{:e}".format(self.optimizer.learning_rate) + "...")
  46. self.metrics = metrics
  48. return
  50. class LossPCLearningRateScheduler(PCLearningRateScheduler):
  51. def __init__(self, optimizer, factor = 0.8, patience = 2, threshold = 1e-4, cooldown = 2, min_learning_rate = 1e-6):
  52. super().__init__(optimizer, factor, patience, threshold, cooldown, min_learning_rate)
  54. return
  56. def step(self, metrics):
  57. if self.metrics is None:
  58. self.metrics = metrics
  59. return
  61. if self.optimizer.learning_rate < self.min_learning_rate:
  62. self.metrics = metrics
  63. return
  65. if self.cooldown_counter > 0:
  66. self.cooldown_counter -= 1
  67. self.metrics = metrics
  68. return
  70. if abs(self.metrics - metrics) < self.threshold:
  71. if self.patience_counter < self.patience:
  72. self.patience_counter += 1
  73. self.metrics = metrics
  74. return
  76. self.optimizer.learning_rate *= self.factor
  77. logger.log_info("Reducing PC learning rate to " + "{:e}".format(self.optimizer.learning_rate) + "...")
  79. self.cooldown_counter = self.cooldown
  80. self.patience_counter = 0
  81. else:
  82. self.patience_counter = 0
  84. self.metrics = metrics
  86. return
  88. class PCOptimizer:
  89. def __init__(self, pc_joint, pc_marginal, device = torch.device("cuda"), learning_rate = 1e-1, prior_factor = 1e2, projection_epsilon = 1e-2):
  90. self.device = device
  91. self.learning_rate = learning_rate
  92. self.smoothing_epsilon = torch.finfo(torch.float).eps
  93. self.prior_factor = prior_factor
  94. self.projection_epsilon = projection_epsilon
  95. self.pc_joint = pc_joint
  96. self.pc_marginal = pc_marginal
  97. self.weights_prior = None
  99. return
  101. def set_weights_prior(self, weights_prior):
  102. self.weights_prior = weights_prior
  104. for i in range(len(self.weights_prior)):
  105. self.weights_prior[i] *= self.prior_factor
  107. return
  109. @abc.abstractmethod
  110. def step(self, matrix_pc = None, matrix_neural = None, matrix_npc = None, labels_class = None):
  111. pass
  113. class CCCPPCOptimizer(PCOptimizer):
  114. def __init__(self, pc_joint, pc_marginal, device = torch.device("cuda"), learning_rate = 1e-1, prior_factor = 1e2, projection_epsilon = 1e-2):
  115. super().__init__(pc_joint, pc_marginal, device, learning_rate, prior_factor, projection_epsilon)
  117. return
  119. def step(self, matrix_pc = None, matrix_neural = None, matrix_npc = None, labels_class = None):
  120. self.pc_joint.reuse_forward = False
  121. self.pc_marginal.reuse_forward = False
  123. for sum_node in self.pc_joint.sum_nodes:
  124. child_values_forward = []
  125. weight_normalization_sum_node = 0
  127. for child in sum_node.children:
  128. child_values_forward.append(child.value_forward)
  130. child_values_forward = torch.stack(child_values_forward) # number of children x batch size
  131. weight_updates = torch.exp(sum_node.value_backward + child_values_forward - self.pc_joint.root_node.value_forward) # number of children x batch size
  132. weight_updates = torch.sum(weight_updates, 1) # number of children
  133. weight_updates = torch.unsqueeze(weight_updates, 1) # number of children x 1
  134. sum_node.weights *= weight_updates # number of children x 1
  136. # Local weight normalization with Laplace smoothing
  137. for weight_sum_node in sum_node.weights:
  138. weight_normalization_sum_node += weight_sum_node + self.smoothing_epsilon
  140. sum_node.weights = (sum_node.weights + self.smoothing_epsilon) / weight_normalization_sum_node
  142. self.pc_marginal.set_weights(self.pc_joint.get_weights())
  144. return
  146. class PGDPCOptimizer(PCOptimizer):
  147. def __init__(self, pc_joint, pc_marginal, device = torch.device("cuda"), learning_rate = 1e-1, prior_factor = 1e2, projection_epsilon = 1e-2):
  148. super().__init__(pc_joint, pc_marginal, device, learning_rate, prior_factor, projection_epsilon)
  150. return
  152. def step(self, matrix_pc = None, matrix_neural = None, matrix_npc = None, labels_class = None):
  153. self.pc_joint.reuse_forward = False
  154. self.pc_marginal.reuse_forward = False
  156. counter_sum_node = 0
  157. matrix_pc = matrix_pc.to(self.device)
  158. matrix_neural = matrix_neural.to(self.device)
  159. matrix_npc_transposed = matrix_npc.t() # number of class labels x batch size
  160. matrix_npc_transposed = matrix_npc_transposed.to(self.device)
  161. labels_class = labels_class.to(self.device)
  162. progress_bar = None
  164. if self.device == torch.device("cpu"):
  165. progress_bar = tqdm.tqdm(total = len(self.pc_joint.sum_nodes), leave = False)
  166. progress_bar.set_description_str("[INFO]: Optimizing PC")
  168. for (sum_node_joint, sum_node_marginal, weights_prior) in zip(self.pc_joint.sum_nodes, self.pc_marginal.sum_nodes, self.weights_prior):
  169. if progress_bar is not None:
  170. progress_bar.n = counter_sum_node + 1
  171. progress_bar.refresh()
  172. counter_sum_node += 1
  174. for (i, (child_joint, child_marginal)) in enumerate(zip(sum_node_joint.children, sum_node_marginal.children)):
  175. # Compute weight updates in log space
  176. weight_updates_joint = torch.exp(sum_node_joint.value_backward + child_joint.value_forward - self.pc_joint.root_node.value_forward)
  177. weight_updates_marginal = torch.exp(sum_node_marginal.value_backward + child_marginal.value_forward - self.pc_marginal.root_node.value_forward)
  179. # Set gradients corresponding to zero root node forward values to zero
  180. mask_1_joint = ((sum_node_joint.value_backward + child_joint.value_forward) == -float("inf"))
  181. mask_2_joint = (self.pc_joint.root_node.value_forward == -float("inf"))
  182. mask_joint = mask_1_joint & mask_2_joint
  183. mask_1_marginal = ((sum_node_marginal.value_backward + child_marginal.value_forward) == -float("inf"))
  184. mask_2_marginal = (self.pc_marginal.root_node.value_forward == -float("inf"))
  185. mask_marginal = mask_1_marginal & mask_2_marginal
  186. weight_updates_joint[mask_joint] = 0
  187. weight_updates_marginal[mask_marginal] = 0
  189. weight_updates = weight_updates_joint - weight_updates_marginal
  190. weight_updates = weight_updates.reshape(matrix_pc.shape) # number of class labels x product of category size of all attributes
  191. weight_updates *= matrix_pc # number of class labels x product of category size of all attributes
  192. weight_updates = weight_updates.t() # product of category size of all attributes x number of class labels
  193. weight_updates = torch.index_select(weight_updates, 1, labels_class) # product of category size of all attributes x batch size
  194. weight_updates *= matrix_neural # product of category size of all attributes x batch size
  195. weight_updates = torch.sum(weight_updates, 0) # 1 x batch size
  196. weight_updates /= matrix_npc_transposed[labels_class, torch.arange(matrix_npc_transposed.shape[1])] # 1 x batch size
  198. # Average weight updates with Dirichlet prior
  199. weight_updates_count = weight_updates.shape[0]
  200. weight_updates = torch.sum(weight_updates, 0, keepdim = True)
  201. weight_updates += (weights_prior[i] - 1) / sum_node_joint.weights[i]
  202. weight_updates /= weight_updates_count
  204. sum_node_joint.weights[i] += self.learning_rate * weight_updates
  206. if sum_node_joint.weights[i] <= 0:
  207. sum_node_joint.weights[i] = self.projection_epsilon
  209. if progress_bar is not None:
  210. progress_bar.close()
  212. self.pc_marginal.set_weights(self.pc_joint.get_weights())
  214. return
  216. class Node:
  217. def __init__(self):
  218. self.children = []
  219. self.depth = None
  220. self.device = None
  221. self.id = None
  222. self.parents = []
  223. self.value_backward = None
  224. self.value_forward = None
  226. return
  228. def backward(self):
  229. if len(self.parents) == 0:
  230. logger.log_fatal("Node " + str(self.id) + " has no parents. Quit.")
  231. exit(-1)
  233. value_backward_parents_product = []
  234. value_backward_parents_sum = []
  235. value_backward_product = None
  236. value_backward_sum = None
  237. value_forward_parents_product = []
  238. weights_parents_sum = []
  240. for parent in self.parents:
  241. if isinstance(parent, ProductNode):
  242. value_backward_parents_product.append(parent.value_backward)
  243. value_forward_parents_product.append(parent.value_forward)
  244. elif isinstance(parent, SumNode):
  245. value_backward_parents_sum.append(parent.value_backward)
  246. weights_parents_sum.append(parent.weights[parent.weights_index_by_child_id[self.id]])
  248. if len(value_backward_parents_product) > 0:
  249. value_backward_parents_product = torch.stack(value_backward_parents_product)
  251. if len(value_backward_parents_sum) > 0:
  252. value_backward_parents_sum = torch.stack(value_backward_parents_sum)
  254. if len(value_forward_parents_product) > 0:
  255. value_forward_parents_product = torch.stack(value_forward_parents_product)
  257. if len(weights_parents_sum) > 0:
  258. weights_parents_sum = torch.stack(weights_parents_sum)
  259. weights_parents_sum = torch.Tensor(weights_parents_sum).reshape(-1, 1)
  260. weights_parents_sum = weights_parents_sum.to(self.device)
  262. if len(value_backward_parents_product) > 0:
  263. value_backward_product = value_backward_parents_product + value_forward_parents_product - self.value_forward
  265. if len(value_backward_parents_sum) > 0:
  266. value_backward_sum = value_backward_parents_sum + torch.log(weights_parents_sum)
  268. if value_backward_product is not None and value_backward_sum is None:
  269. self.value_backward = value_backward_product
  270. elif value_backward_product is None and value_backward_sum is not None:
  271. self.value_backward = value_backward_sum
  272. else:
  273. self.value_backward = torch.stack([value_backward_product, value_backward_sum])
  275. # Compute backward values in log space
  276. # Log-Sum-Exp trick: https://gregorygundersen.com/blog/2020/02/09/log-sum-exp/
  277. value_backward_max = torch.max(self.value_backward, 0)[0]
  278. self.value_backward -= value_backward_max
  279. self.value_backward = torch.exp(self.value_backward)
  280. self.value_backward = torch.sum(self.value_backward, 0)
  281. self.value_backward = torch.log(self.value_backward) + value_backward_max
  283. return
  285. @abc.abstractmethod
  286. def forward(self):
  287. pass
  289. class CategoricalLeafNode(Node):
  290. def __init__(self):
  291. super().__init__()
  293. self.attribute_index = None
  294. self.category_index = None
  296. return
  298. def forward(self):
  299. return
  301. def set_binary(self, settings):
  302. if self.attribute_index < 0 or self.category_index < 0:
  303. logger.log_fatal("Invalid categorical leaf node. Quit.")
  304. exit(-1)
  306. categories = settings[self.attribute_index]
  307. self.value_forward = categories[:, self.category_index].float()
  308. self.value_forward = self.value_forward.to(self.device)
  310. # Compute forward values in log space
  311. self.value_forward = torch.log(self.value_forward)
  313. return
  315. def set_categorical(self, settings):
  316. if self.attribute_index < 0 or self.category_index < 0:
  317. logger.log_fatal("Invalid categorical leaf node. Quit.")
  318. exit(-1)
  320. variables = settings[:, self.attribute_index]
  321. settings = (variables == self.category_index)
  322. settings_marginal = (variables < 0)
  324. self.value_forward = torch.logical_or(settings, settings_marginal).float()
  325. self.value_forward = self.value_forward.to(self.device)
  327. # Compute forward values in log space
  328. self.value_forward = torch.log(self.value_forward)
  330. return
  332. class ProductNode(Node):
  333. def __init__(self):
  334. super().__init__()
  336. return
  338. def forward(self):
  339. if len(self.children) == 0:
  340. logger.log_fatal("Product node " + str(self.id) + " has no children. Quit.")
  341. exit(-1)
  343. value_forward_children = []
  345. for child in self.children:
  346. value_forward_children.append(child.value_forward)
  348. # Compute forward values in log space
  349. value_forward_children = torch.stack(value_forward_children)
  350. self.value_forward = torch.sum(value_forward_children, 0)
  352. return
  354. class SumNode(Node):
  355. def __init__(self):
  356. super().__init__()
  358. self.leaf = False
  359. self.weights = []
  360. self.weights_index_by_child_id = {}
  362. return
  364. def forward(self):
  365. if len(self.children) == 0:
  366. logger.log_fatal("Sum node " + str(self.id) + " has no children. Quit.")
  367. exit(-1)
  369. value_forward_children = []
  371. for child in self.children:
  372. value_forward_children.append(child.value_forward)
  374. # Compute forward values in log space
  375. # Log-Sum-Exp trick: https://gregorygundersen.com/blog/2020/02/09/log-sum-exp/
  376. value_forward_children = torch.stack(value_forward_children)
  377. value_forward_children_max = torch.max(value_forward_children, 0)[0]
  378. value_forward_children_max[value_forward_children_max == -float("inf")] = 0
  379. value_forward_children -= value_forward_children_max
  380. value_forward_children = torch.exp(value_forward_children)
  381. value_forward_children *= self.weights
  382. self.value_forward = torch.sum(value_forward_children, 0)
  383. self.value_forward = torch.log(self.value_forward) + value_forward_children_max
  385. return
  387. class ProbabilisticCircuit:
  388. def __init__(self, device = torch.device("cuda")):
  389. self.batch_size = -1
  390. self.depth = None
  391. self.device = device
  392. self.induced_trees = []
  393. self.leaf_nodes = []
  394. self.leaf_nodes_dict = {}
  395. self.nodes = []
  396. self.product_nodes = []
  397. self.reuse_backward = False
  398. self.reuse_forward = False
  399. self.root_node = None
  400. self.sum_nodes = []
  401. self.traversal_order_backward = []
  402. self.traversal_order_forward = []
  404. return
  406. def __call__(self, settings, categorical = True):
  407. if categorical:
  408. self.set_leaf_nodes_categorical(settings)
  409. else:
  410. self.set_leaf_nodes_binary(settings)
  412. return self.forward()
  414. def backward(self):
  415. if not self.reuse_backward:
  416. if len(self.traversal_order_backward) == 0:
  417. logger.log_fatal("Empty tree. Quit.")
  418. exit(-1)
  420. if self.root_node is None or self.traversal_order_backward[0][0].id != self.root_node.id:
  421. logger.log_fatal("Missing root node. Quit.")
  422. exit(-1)
  424. if self.root_node.value_forward is None:
  425. logger.log_fatal("Missing root node forward value. Quit.")
  426. exit(-1)
  428. self.reuse_backward = True
  430. # Initialize root node backward value in log space
  431. self.root_node.value_backward = torch.log(torch.ones(self.batch_size))
  432. self.root_node.value_backward = self.root_node.value_backward.to(self.device)
  434. if self.device == torch.device("cpu"):
  435. counter_node = 0
  436. progress_bar = tqdm.tqdm(total = len(self.nodes), leave = False)
  437. progress_bar.set_description_str("[INFO]: Running PC backward pass")
  439. for level in self.traversal_order_backward[1:]:
  440. for node in level:
  441. progress_bar.n = counter_node + 1
  442. progress_bar.refresh()
  443. counter_node += 1
  445. node.backward()
  447. progress_bar.close()
  448. else:
  449. for level in self.traversal_order_backward[1:]:
  450. for node in level:
  451. node.backward()
  453. return
  455. def forward(self):
  456. if not self.reuse_forward:
  457. if len(self.traversal_order_forward) == 0:
  458. logger.log_fatal("Empty tree. Quit.")
  459. exit(-1)
  461. if self.root_node is None or self.traversal_order_forward[-1][0].id != self.root_node.id:
  462. logger.log_fatal("Missing root node. Quit.")
  463. exit(-1)
  465. self.reuse_backward = False
  466. self.reuse_forward = True
  468. if self.device == torch.device("cpu"):
  469. counter_node = 0
  470. progress_bar = tqdm.tqdm(total = len(self.nodes), leave = False)
  471. progress_bar.set_description_str("[INFO]: Running PC forward pass")
  473. for level in self.traversal_order_forward:
  474. for node in level:
  475. progress_bar.n = counter_node + 1
  476. progress_bar.refresh()
  477. counter_node += 1
  479. node.forward()
  481. progress_bar.close()
  482. else:
  483. for level in self.traversal_order_forward:
  484. for node in level:
  485. node.forward()
  487. return self.root_node.value_forward
  489. def gather_induced_trees(self):
  490. for induced_tree in self.recurse_induced_trees(self.root_node):
  491. induced_tree[1] = numpy.prod(induced_tree[1])
  492. self.induced_trees.append(induced_tree)
  494. return
  496. def get_weights(self):
  497. weights = []
  499. for sum_node in self.sum_nodes:
  500. weights.append(torch.clone(sum_node.weights))
  502. return weights
  504. def load(self, file_path_pc):
  505. if not os.path.exists(file_path_pc):
  506. logger.log_fatal("Invalid PC file path. Quit.")
  507. exit(-1)
  509. self.reuse_backward = False
  510. self.reuse_forward = False
  512. with open(file_path_pc, "r") as file_pc:
  513. categorical_leaf_node_id = -1
  514. counter_line = 0
  515. id_to_nodes = {}
  516. lines = file_pc.readlines()
  517. progress_bar = tqdm.tqdm(total = len(lines), leave = False)
  518. reading_nodes = True
  520. progress_bar.set_description_str("[INFO]: Loading PC")
  522. for line in lines:
  523. progress_bar.n = counter_line + 1
  524. progress_bar.refresh()
  525. counter_line += 1
  527. line = line.strip()
  529. if line[0] == "#":
  530. line = line.replace("#", "")
  532. if line == "NODES":
  533. reading_nodes = True
  534. elif line == "EDGES":
  535. reading_nodes = False
  537. continue
  539. line_list = line.split(",")
  541. if reading_nodes:
  542. node_id = int(line_list[0])
  543. node_type = line_list[1]
  545. if node_type == "SUM":
  546. sum_node = SumNode()
  547. sum_node.device = self.device
  548. sum_node.id = node_id
  549. self.nodes.append(sum_node)
  550. id_to_nodes[sum_node.id] = sum_node
  551. self.sum_nodes.append(sum_node)
  552. elif node_type == "PRD":
  553. product_node = ProductNode()
  554. product_node.device = self.device
  555. product_node.id = node_id
  556. self.nodes.append(product_node)
  557. id_to_nodes[product_node.id] = product_node
  558. self.product_nodes.append(product_node)
  559. elif node_type == "CatNode" or node_type == "CATNODE":
  560. categorical_leaf_node_list = []
  561. node_attribute_index = int(line_list[2])
  562. node_probabilities = line_list[3:]
  564. for i in range(0, len(node_probabilities)):
  565. node_probabilities[i] = float(node_probabilities[i])
  567. if node_attribute_index in self.leaf_nodes_dict.keys():
  568. categorical_leaf_node_list = self.leaf_nodes_dict[node_attribute_index]
  569. else:
  570. for category_index in range(0, len(node_probabilities)):
  571. categorical_leaf_node = CategoricalLeafNode()
  572. categorical_leaf_node.attribute_index = node_attribute_index
  573. categorical_leaf_node.category_index = category_index
  574. categorical_leaf_node.device = self.device
  575. categorical_leaf_node.id = categorical_leaf_node_id
  576. categorical_leaf_node_id -= 1
  578. categorical_leaf_node_list.append(categorical_leaf_node)
  579. self.leaf_nodes.append(categorical_leaf_node)
  580. self.nodes.append(categorical_leaf_node)
  581. id_to_nodes[categorical_leaf_node.id] = categorical_leaf_node
  583. self.leaf_nodes_dict[node_attribute_index] = categorical_leaf_node_list
  585. sum_node = SumNode()
  586. sum_node.children = categorical_leaf_node_list
  587. sum_node.device = self.device
  588. sum_node.id = node_id
  589. sum_node.leaf = True
  590. sum_node.weights = node_probabilities
  591. self.nodes.append(sum_node)
  592. id_to_nodes[sum_node.id] = sum_node
  593. self.sum_nodes.append(sum_node)
  595. for (i, categorical_leaf_node) in enumerate(categorical_leaf_node_list):
  596. sum_node.weights_index_by_child_id[categorical_leaf_node.id] = i
  597. categorical_leaf_node.parents.append(sum_node)
  598. elif node_type == "CATNODEPRD":
  599. node_attribute_index = int(line_list[2])
  600. node_category_index = int(line_list[3])
  602. categorical_leaf_node = CategoricalLeafNode()
  603. categorical_leaf_node.attribute_index = node_attribute_index
  604. categorical_leaf_node.category_index = node_category_index
  605. categorical_leaf_node.device = self.device
  606. categorical_leaf_node.id = node_id
  608. self.leaf_nodes.append(categorical_leaf_node)
  609. self.nodes.append(categorical_leaf_node)
  610. id_to_nodes[categorical_leaf_node.id] = categorical_leaf_node
  611. else:
  612. nodes = []
  613. node_id_first = int(line_list[0])
  614. node_id_second = int(line_list[1])
  616. nodes.append(id_to_nodes[node_id_first])
  617. nodes.append(id_to_nodes[node_id_second])
  619. if len(nodes) != 2:
  620. logger.log_fatal("Invalid edge. Quit.")
  621. exit(-1)
  623. if len(line_list) >= 3:
  624. node_weight = float(line_list[2])
  626. if isinstance(nodes[0], SumNode) and not nodes[0].leaf:
  627. nodes[0].children.append(nodes[1])
  628. nodes[0].weights.append(node_weight)
  629. nodes[0].weights_index_by_child_id[nodes[1].id] = len(nodes[0].weights) - 1
  630. nodes[1].parents.append(nodes[0])
  631. elif isinstance(nodes[1], SumNode) and not nodes[0].leaf:
  632. nodes[1].children.append(nodes[0])
  633. nodes[1].weights.append(node_weight)
  634. nodes[1].weights_index_by_child_id[nodes[0].id] = len(nodes[1].weights) - 1
  635. nodes[0].parents.append(nodes[1])
  636. else:
  637. if isinstance(nodes[0], ProductNode):
  638. nodes[0].children.append(nodes[1])
  639. nodes[1].parents.append(nodes[0])
  640. elif isinstance(nodes[1], ProductNode):
  641. nodes[1].children.append(nodes[0])
  642. nodes[0].parents.append(nodes[1])
  644. progress_bar.close()
  646. for sum_node in self.sum_nodes:
  647. sum_node.weights = torch.Tensor(sum_node.weights).reshape(-1, 1)
  648. sum_node.weights = sum_node.weights.to(self.device)
  650. root_nodes = []
  652. for node in self.nodes:
  653. if len(node.parents) == 0:
  654. root_nodes.append(node)
  656. if len(root_nodes) != 1:
  657. logger.log_fatal("Invalid PC. Quit.")
  658. exit(-1)
  660. self.depth = self.traverse({0: root_nodes}, 0)
  661. self.root_node = root_nodes[0]
  662. self.traversal_order_backward = self.topological_sort_backward()
  663. self.traversal_order_forward = self.traversal_order_backward.copy()
  664. self.traversal_order_forward.reverse()
  666. def getLengthNestedList(nested_list):
  667. length = 0
  669. for item in nested_list:
  670. if isinstance(item, list):
  671. length += getLengthNestedList(item)
  672. else:
  673. length += 1
  675. return length
  677. if getLengthNestedList(self.traversal_order_backward) != len(self.nodes):
  678. logger.log_fatal("Invalid PC backward traversal. Quit.")
  679. exit(-1)
  681. if getLengthNestedList(self.traversal_order_forward) != len(self.nodes):
  682. logger.log_fatal("Invalid PC forward traversal. Quit.")
  683. exit(-1)
  685. return
  687. def normalize_weights(self, smoothing_epsilon):
  688. for sum_node in self.sum_nodes:
  689. value_forward_children = []
  690. weight_normalization = 0
  692. for child in sum_node.children:
  693. value_forward_children.append(child.value_forward)
  695. value_forward_children = torch.stack(value_forward_children)
  696. value_forward_children_max = torch.max(value_forward_children, 0)[0]
  698. for (i, child) in enumerate(sum_node.children):
  699. weight_normalization += sum_node.weights[i] * torch.exp(child.value_forward - value_forward_children_max) + smoothing_epsilon
  701. # Local weight normalization with Laplace smoothing
  702. for (i, child) in enumerate(sum_node.children):
  703. weight = sum_node.weights[i] * torch.exp(child.value_forward - value_forward_children_max) + smoothing_epsilon
  704. sum_node.weights[i] = weight / weight_normalization
  706. return
  708. def randomize_weights(self):
  709. self.reuse_backward = False
  710. self.reuse_forward = False
  712. weights = self.get_weights()
  714. for i in range(len(weights)):
  715. weights[i] = weights[i].uniform_(0, 1)
  716. weights[i] /= torch.sum(weights[i])
  718. self.set_weights(weights)
  720. return
  722. def recurse_induced_trees(self, node):
  723. if isinstance(node, SumNode):
  724. for (i, child) in enumerate(node.children):
  725. for sub_induced_tree in self.recurse_induced_trees(child):
  726. induced_tree = [{node}, [node.weights[i].item()]]
  727. induced_tree[0].update(sub_induced_tree[0])
  728. induced_tree[1] += sub_induced_tree[1]
  729. yield induced_tree
  730. elif isinstance(node, ProductNode):
  731. sub_induced_trees_product = []
  733. for child in node.children:
  734. sub_induced_trees = []
  735. for sub_induced_tree in self.recurse_induced_trees(child):
  736. sub_induced_trees.append(sub_induced_tree)
  737. sub_induced_trees_product.append(sub_induced_trees)
  739. for sub_induced_trees in itertools.product(*sub_induced_trees_product):
  740. induced_tree = [{node}, []]
  741. for sub_induced_tree in sub_induced_trees:
  742. induced_tree[0].update(sub_induced_tree[0])
  743. induced_tree[1] += sub_induced_tree[1]
  744. yield induced_tree
  745. elif isinstance(node, CategoricalLeafNode):
  746. yield [{node}, []]
  748. return
  750. def set_leaf_nodes_binary(self, settings):
  751. self.reuse_backward = False
  752. self.reuse_forward = False
  753. self.batch_size = settings[0].shape[0]
  755. for leaf_node in self.leaf_nodes:
  756. leaf_node.set_binary(settings)
  758. return
  760. def set_leaf_nodes_categorical(self, settings):
  761. self.reuse_backward = False
  762. self.reuse_forward = False
  763. self.batch_size = settings.shape[0]
  765. for leaf_node in self.leaf_nodes:
  766. leaf_node.set_categorical(settings)
  768. return
  770. def set_weights(self, weights):
  771. if len(weights) != len(self.sum_nodes):
  772. logger.log_fatal("Invalid weights. Quit.")
  773. exit(-1)
  775. self.reuse_backward = False
  776. self.reuse_forward = False
  778. for (sum_node, sum_node_weights) in zip(self.sum_nodes, weights):
  779. sum_node.weights = torch.clone(sum_node_weights).to(self.device)
  781. return
  783. def topological_sort_backward(self):
  784. levels = []
  785. parent_count = {}
  786. queue = []
  788. for node in self.nodes:
  789. parent_count[node.id] = len(node.parents)
  791. queue.append(self.root_node)
  793. while len(queue) > 0:
  794. level = []
  795. queue_next_level = []
  797. for node in queue:
  798. level.append(node)
  800. for child in node.children:
  801. parent_count[child.id] -= 1
  803. if parent_count[child.id] <= 0:
  804. queue_next_level.append(child)
  806. levels.append(level)
  807. queue = queue_next_level.copy()
  809. return levels
  811. def traverse(self, layers, depth):
  812. if depth not in layers.keys():
  813. return depth - 1
  815. for node in layers[depth]:
  816. node.depth = depth
  818. for child in node.children:
  819. if depth + 1 not in layers.keys():
  820. layers[depth + 1] = []
  822. layers[depth + 1].append(child)
  824. depth_next = self.traverse(layers, depth + 1)
  826. if depth_next > depth:
  827. depth = depth_next
  829. return depth

pc.py at commit 4a46c28, under CC-BY-NC-SA-4.0 · at the source

Overview

Authors: Weixin Chen1, Simon Yu2, Huajie Shao3, Lui Sha1, Han Zhao1
  1. Department of Computer Science, University of Illinois Urbana-Champaign, Urbana, IL USA
  2. Department of Electrical and Computer Engineering, University of Illinois Urbana-Champaign, Urbana, IL USA
  3. Department of Computer Science, College of William and Mary, Williamsburg, VA USA
Institutions: University of Illinois Urbana-Champaign (United States); William & Mary (United States)
Journal: Machine learning, volume 115, issue 9, article 207
Dates: received 12 January 2025; accepted 2 July 2026; published online 29 August 2026; in print 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1007/s10994-026-07118-7 · PMID 42670317 · PMCID PMC13525958 · OpenAlex W4406692178
Open access: hybrid, a free copy (OpenAlex)
Preprint: arxiv:2501.07021
Status: code verified
Categories: systems (subfield)
Keywords: Interpretability, Compositional models, Probabilistic circuits, Logical reasoning
Topic: Neural Networks and Applications (Artificial Intelligence, Computer Science), according to OpenAlex
Funding: National Science Foundation (2311085, #80NSSC22M0070); National Aeronautics and Space Administration (80NSSC22M0070); Division of Electrical, Communications and Cyber Systems
Citations: not cited yet (Europe PMC); 109 references in the paper

Abstract

End-to-end deep neural networks have achieved remarkable success across various domains but are often criticized for their lack of interpretability. While post hoc explanation methods attempt to address this issue, they often fail to accurately represent these black-box models, resulting in misleading or incomplete explanations. To overcome these challenges, we propose an inherently transparent model architecture called Neural Probabilistic Circuits (NPCs), which enable compositional and interpretable predictions through logical reasoning. In particular, an NPC consists of two modules: an attribute recognition model, which predicts probabilities for various attributes, and a task predictor built on a probabilistic circuit, which enables logical reasoning over recognized attributes to make class predictions. To train NPCs, we introduce a three-stage training algorithm comprising attribute recognition, circuit construction, and joint optimization. Moreover, we theoretically demonstrate that an NPC’s error is upper-bounded by a linear combination of the errors from its modules. To further demonstrate the interpretability of NPC, we provide both the most probable explanations and the counterfactual explanations. Empirical results on four benchmark datasets show that NPCs strike a balance between interpretability and performance, achieving results competitive even with those of end-to-end black-box models while providing enhanced interpretability.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 8 matches between paragraphs and lines of code.

uiuctml/npc-models

License: CC-BY-NC-SA-4.0
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 4a46c28f7cbfcc4fd89c5460de74c573fd14002c, 9 October 2025
Languages: Python (18), Shell (10)
Size: 36 files, 28 scripts
Software Heritage: not archived
Found in: “Code Availability”
Holds: README, license file, environment (requirements.txt), tests, documentation
Not found: CITATION.cff, continuous integration
Tools: PyTorch (14 files), NumPy (2 files), Pillow (1 file), scikit-learn (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
30 files
At the source: github.com/uiuctml/npc-models

Code Availability

The code is available at https://github.com/uiuctml/npc-models.

Reproduced under the paper's license (CC BY), from the paper cited above.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 28 scripts, each with its path and the digest of its content;
  • 8 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Data Availability

We employed four publicly available image datasets in this work, namely, Animal with Attributes 2 (AwA2), CelebFaces Attributes (CelebA), German Traffic Sign Recognition Benchmark (GTSRB), and Modified National Institute of Standards and Technology (MNIST). AwA2 is available at https://cvml.ista.ac.at/AwA2/. CelebA is available at https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html. GTSRB is availble at https://www.kaggle.com/datasets/meowmeowmeowmeowmeow/gtsrb-german-traffic-sign. MNIST is available at https://huggingface.co/datasets/ylecun/mnist. Three out of the four datasets, namely, AwA2, GTSRB, and MNIST, grants us rights to freely use, redistribute, and publish portions of the datasets for research purposes. CelebA, albeit granting limited access for non-commercial research purposes, explicitly forbids redistributions or publications of any portion of the dataset, as per its terms of use. Therefore, all instances of CelebA images are redacted from this work in compliance with such terms of use. During our experiments, we have generated additional data for all four datasets, such as annotations in regards to attribute recognition. Upon acceptance, all data generated during our experiments will be released and remain publicly available under appropriate licenses.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 2, 28 September 2026

  • Publisher: n/a → Springer Science+Business Media
  • Funding: added National Science Foundation: 2311085, #80NSSC22M0070; National Aeronautics and Space Administration: 80NSSC22M0070; Division of Electrical, Communications and Cyber Systems

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 5 authors, 4 keywords, 12 references.

Cite

This paper

Chen, W., Yu, S., Shao, H., Sha, L., & Zhao, H. (2026). Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions Through Logical Reasoning. Machine learning, 115(9), 207. https://doi.org/10.1007/s10994-026-07118-7

BibTeX

@article{chen2026neural,
author = {Chen, Weixin and Yu, Simon and Shao, Huajie and Sha, Lui and Zhao, Han},
title = {{Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions Through Logical Reasoning}},
journal = {Machine learning},
year = {2026},
month = aug,
volume = {115},
number = {9},
pages = {207},
publisher = {Springer Science+Business Media},
issn = {0885-6125},
doi = {10.1007/s10994-026-07118-7},
url = {https://doi.org/10.1007/s10994-026-07118-7},
pmid = {42670317},
pmcid = {PMC13525958}
}

RIS

TY - JOUR
AU - Chen, Weixin
AU - Yu, Simon
AU - Shao, Huajie
AU - Sha, Lui
AU - Zhao, Han
TI - Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions Through Logical Reasoning
T2 - Machine learning
J2 - Mach Learn
PY - 2026
DA - 2026/08/29
VL - 115
IS - 9
SP - 207
SN - 0885-6125
PB - Springer Science+Business Media
DO - 10.1007/s10994-026-07118-7
UR - https://doi.org/10.1007/s10994-026-07118-7
LA - en
ER -

CSL-JSON

{
"id": "10.1007/s10994-026-07118-7",
"type": "article-journal",
"title": "Neural Probabilistic Circuits: Enabling Compositional and Interpretable Predictions Through Logical Reasoning",
"container-title": "Machine learning",
"author": [
{
"family": "Chen",
"given": "Weixin"
},
{
"family": "Yu",
"given": "Simon"
},
{
"family": "Shao",
"given": "Huajie"
},
{
"family": "Sha",
"given": "Lui"
},
{
"family": "Zhao",
"given": "Han"
}
],
"container-title-short": "Mach Learn",
"volume": "115",
"issue": "9",
"page": "207",
"DOI": "10.1007/s10994-026-07118-7",
"PMID": "42670317",
"PMCID": "PMC13525958",
"ISSN": "0885-6125",
"publisher": "Springer Science+Business Media",
"URL": "https://doi.org/10.1007/s10994-026-07118-7",
"language": "en",
"issued": {
"date-parts": [
[
2026,
8,
29
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1093/braincomms/fcag253 [code]
Disease detection and classification in temporal lobe epilepsy: step-wise versus simultaneous AI decision models in a multisite neuroimaging study.
Journal: Brain communications
In common: Pillow, PyTorch, scikit-learn, 1 other tool, 1 reference
[2] doi:10.1177/26331055261460858 [code]
The Geometric Signatures of Brain State Transitions: Recursive Informational Curvature Reveals Hidden Dynamics in Primate Cortex.
Journal: Neuroscience insights
In common: Pillow, scikit-learn, NumPy, systems, 1 reference
[3] doi:10.1016/j.isci.2026.117068 [code]
Directed graph neural networks with partial directed coherence for seizure prediction and epileptogenic network characterization.
Journal: iScience
In common: PyTorch, scikit-learn, NumPy, systems, 1 reference
[4] doi:10.1371/journal.pcbi.1014656 [code]
Contrastive learning to fine-tune feature extraction models for the visual cortex.
Journal: PLoS computational biology
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[5] doi:10.1038/s41593-026-02388-9 [code]
Hippocampal CA3 connectomics reveals a gradient of mossy fiber inputs and selective feedforward inhibition onto pyramidal cells.
Journal: Nature neuroscience
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[6] doi:10.1523/jneurosci.0038-26.2026 [code]
Multidimensional Feature Tuning in Category Selective Areas of Human Visual Cortex.
Journal: The Journal of neuroscience : the official journal of the Society for Neuroscience
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[7] doi:10.1016/j.isci.2026.116825 [code]
Social hierarchy shapes behavioral and transcriptional responses to chronic stress and ketamine in male mice.
Journal: iScience
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[8] doi:10.1038/s41467-026-75347-4 [code]
Sleep reveals dynamics integrating and segregating movement and stimulus representations in V1.
Journal: Nature communications
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[9] doi:10.1016/j.isci.2026.116206 [code]
Gut distension evokes rapid neural dynamics in vagal and hindbrain populations of larval zebrafish.
Journal: iScience
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems
[10] doi:10.1371/journal.pbio.3003824 [code]
Flexible goal learning involves coordinated population activity in dCA1 and medial orbitofrontal cortex.
Journal: PLoS biology
In common: Pillow, PyTorch, scikit-learn, 1 other tool, systems

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.