Convergence-divergence circuits for multimodal integration of innate and learned opponent valences.

Code ↔ Paper

8 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 8 matches

[1] § Materials and methods › UMAP › UMAP generation ↔ umap/umap_.py, lines 1485 to 1737 · score 0.97 · uniform manifold approximation, fuzzy simplicial, min_dist, UMAP algorithm, UMAP embedding, n_neighbors · Report a mismatch
[2] § Materials and methods › UMAP › Clustering on UMAP ↔ umap/umap_.py, lines 1485 to 1737 · score 0.84 · Local density, UMAP embedding, nearest neighbors, high dimensional, NN, inverse · Report a mismatch
[3] § Materials and methods › UMAP › UMAP generation ↔ umap/layouts.py, lines 527 to 658 · score 0.81 · stochastic gradient descent, fuzzy simplicial, low dimensional, metric, optimized, embedding · Report a mismatch
[4] § Materials and methods › Geometry ↔ navis/nbl/nblast_funcs.py, lines 878 to 1016 · score 0.64 · dot product, nearest neighbor, NBLAST, straight, score, query · Report a mismatch
[5] § Materials and methods › Geometry ↔ navis/nbl/synblast_funcs.py, lines 230 to 360 · score 0.61 · target neuron, nearest neighbor, product, NBLAST, score, query · Report a mismatch
[6] § Materials and methods › Subcellular topology › Ranked dendrogram ↔ navis/graph/converters.py, lines 484 to 592 · score 0.56 · adding temporary, edge lengths, branches, tree, node, skeleton · Report a mismatch
[7] § Materials and methods › Geometry ↔ navis/nbl/nblast_funcs.py, lines 878 to 1016 · score 0.55 · score matrix, nearest neighbors, NBLAST, vector, neurons · Report a mismatch
[8] § Materials and methods › Geometry ↔ navis/nbl/synblast_funcs.py, lines 230 to 360 · score 0.54 · score matrix, nearest neighbors, NBLAST, morphological, vector, neurons · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The paper is loaded by your browser, with JavaScript: read it at doi.org or on Europe PMC.

The authors' code

Python · 3,784 lines · 143 KB · BSD-3-Clause · 2 matches

# Author: Leland McInnes <[email hidden]>
#
# License: BSD 3 clause
from __future__ import print_function
import locale
from collections import deque
from warnings import warn
import time
from scipy.optimize import curve_fit
from sklearn.base import BaseEstimator, ClassNamePrefixFeaturesOutMixin
from sklearn.utils import check_array, check_random_state
from sklearn.utils.validation import check_is_fitted
from sklearn.metrics import pairwise_distances
from sklearn.preprocessing import normalize
from sklearn.neighbors import KDTree
from sklearn.decomposition import PCA, TruncatedSVD
try:
import joblib
except ImportError:
# sklearn.externals.joblib is deprecated in 0.21, will be removed in 0.23
from sklearn.externals import joblib
import numpy as np
import scipy.sparse
from scipy.sparse import tril as sparse_tril, triu as sparse_triu
import scipy.sparse.csgraph
import numba
import umap.distances as dist
import umap.sparse as sparse
from umap.utils import (
ts,
csr_unique,
fast_knn_indices,
)
from umap.spectral import spectral_layout, tswspectral_layout
from umap.layouts import (
optimize_layout_euclidean,
optimize_layout_generic,
optimize_layout_inverse,
)
from pynndescent import NNDescent
from pynndescent.distances import named_distances as pynn_named_distances
from pynndescent.sparse import sparse_named_distances as pynn_sparse_named_distances
locale.setlocale(locale.LC_NUMERIC, "C")
INT32_MIN = np.iinfo(np.int32).min + 1
INT32_MAX = np.iinfo(np.int32).max - 1
SMOOTH_K_TOLERANCE = 1e-5
MIN_K_DIST_SCALE = 1e-3
NPY_INFINITY = np.inf
NPY_FLOATMAX = np.finfo(np.float32).max
DISCONNECTION_DISTANCES = {
"correlation": 2,
"cosine": 2,
"hellinger": 1,
"jaccard": 1,
"bit_jaccard": 1,
"dice": 1,
}
def flatten_iter(container):
for i in container:
if isinstance(i, (list, tuple)):
for j in flatten_iter(i):
yield j
else:
yield i
def flattened(container):
return tuple(flatten_iter(container))
def breadth_first_search(adjmat, start, min_vertices):
explored = []
queue = deque([start])
levels = {start: 0}
max_level = np.inf
visited = np.zeros(adjmat.shape[0], dtype=np.bool_)
visited[start] = True
while queue:
node = queue.popleft()
explored.append(node)
if max_level == np.inf and len(explored) > min_vertices:
max_level = max(levels.values())
if levels[node] + 1 < max_level:
neighbors = adjmat[node].indices
for neighbour in neighbors:
if not visited[neighbour]:
queue.append(neighbour)
visited[neighbour] = True
levels[neighbour] = levels[node] + 1
return np.array(explored)
def raise_disconnected_warning(
edges_removed,
vertices_disconnected,
disconnection_distance,
total_rows,
threshold=0.1,
verbose=False,
):
"""A simple wrapper function to avoid large amounts of code repetition."""
if verbose & (vertices_disconnected == 0) & (edges_removed > 0):
print(
f"Disconnection_distance = {disconnection_distance} has removed {edges_removed} edges. "
f"This is not a problem as no vertices were disconnected."
)
elif (vertices_disconnected > 0) & (
vertices_disconnected <= threshold * total_rows
):
warn(
f"A few of your vertices were disconnected from the manifold. This shouldn't cause problems.\n"
f"Disconnection_distance = {disconnection_distance} has removed {edges_removed} edges.\n"
f"It has only fully disconnected {vertices_disconnected} vertices.\n"
f"Use umap.utils.disconnected_vertices() to identify them.",
)
elif vertices_disconnected > threshold * total_rows:
warn(
f"A large number of your vertices were disconnected from the manifold.\n"
f"Disconnection_distance = {disconnection_distance} has removed {edges_removed} edges.\n"
f"It has fully disconnected {vertices_disconnected} vertices.\n"
f"You might consider using find_disconnected_points() to find and remove these points from your data.\n"
f"Use umap.utils.disconnected_vertices() to identify them.",
)
@numba.njit(
locals={
"psum": numba.types.float32,
"lo": numba.types.float32,
"mid": numba.types.float32,
"hi": numba.types.float32,
},
parallel=True,
)
def smooth_knn_dist(distances, k, n_iter=64, local_connectivity=1.0, bandwidth=1.0):
"""Compute a continuous version of the distance to the kth nearest
neighbor. That is, this is similar to knn-distance but allows continuous
k values rather than requiring an integral k. In essence we are simply
computing the distance such that the cardinality of fuzzy set we generate
is k.
Parameters
----------
distances: array of shape (n_samples, n_neighbors)
Distances to nearest neighbors for each sample. Each row should be a
sorted list of distances to a given samples nearest neighbors.
k: float
The number of nearest neighbors to approximate for.
n_iter: int (optional, default 64)
We need to binary search for the correct distance value. This is the
max number of iterations to use in such a search.
local_connectivity: int (optional, default 1)
The local connectivity required -- i.e. the number of nearest
neighbors that should be assumed to be connected at a local level.
The higher this value the more connected the manifold becomes
locally. In practice this should be not more than the local intrinsic
dimension of the manifold.
bandwidth: float (optional, default 1)
The target bandwidth of the kernel, larger values will produce
larger return values.
Returns
-------
knn_dist: array of shape (n_samples,)
The distance to kth nearest neighbor, as suitably approximated.
nn_dist: array of shape (n_samples,)
The distance to the 1st nearest neighbor for each point.
"""
target = np.log2(k) * bandwidth
rho = np.zeros(distances.shape[0], dtype=np.float32)
result = np.zeros(distances.shape[0], dtype=np.float32)
# Neighbours pruned by disconnection_distance (or inf entries of a
# precomputed distance matrix) arrive here as inf. They must be excluded
# from rho and from the MIN_K_DIST_SCALE floor: the mean of a row that
# contains inf is inf, which made sigma inf for every point with a pruned
# neighbour and gave all of its remaining edges membership strength 1.0.
flat_distances = distances.ravel()
finite_distances = flat_distances[np.isfinite(flat_distances)]
if finite_distances.shape[0] > 0:
mean_distances = np.mean(finite_distances)
else:
mean_distances = 0.0
for i in numba.prange(distances.shape[0]):
lo = 0.0
hi = NPY_FLOATMAX
mid = 1.0
# TODO: This is very inefficient, but will do for now. FIXME
ith_distances = distances[i]
finite_ith_distances = ith_distances[np.isfinite(ith_distances)]
non_zero_dists = finite_ith_distances[finite_ith_distances > 0.0]
if non_zero_dists.shape[0] >= local_connectivity:
index = int(np.floor(local_connectivity))
interpolation = local_connectivity - index
if index > 0:
rho[i] = non_zero_dists[index - 1]
if interpolation > SMOOTH_K_TOLERANCE:
rho[i] += interpolation * (
non_zero_dists[index] - non_zero_dists[index - 1]
)
else:
rho[i] = interpolation * non_zero_dists[0]
elif non_zero_dists.shape[0] > 0:
rho[i] = np.max(non_zero_dists)
for n in range(n_iter):
psum = 0.0
for j in range(1, distances.shape[1]):
d = distances[i, j] - rho[i]
if d > 0:
psum += np.exp(-(d / mid))
else:
psum += 1.0
if np.fabs(psum - target) < SMOOTH_K_TOLERANCE:
break
if psum > target:
hi = mid
mid = (lo + hi) / 2.0
else:
lo = mid
if hi >= NPY_FLOATMAX:
mid *= 2
else:
mid = (lo + hi) / 2.0
result[i] = mid
# TODO: This is very inefficient, but will do for now. FIXME
if rho[i] > 0.0:
mean_ith_distances = np.mean(finite_ith_distances)
if result[i] < MIN_K_DIST_SCALE * mean_ith_distances:
result[i] = MIN_K_DIST_SCALE * mean_ith_distances
else:
if result[i] < MIN_K_DIST_SCALE * mean_distances:
result[i] = MIN_K_DIST_SCALE * mean_distances
return result, rho
def nearest_neighbors(
X,
n_neighbors,
metric,
metric_kwds,
angular,
random_state,
low_memory=True,
use_pynndescent=True,
n_jobs=-1,
verbose=False,
):
"""Compute the ``n_neighbors`` nearest points for each data point in ``X``
under ``metric``. This may be exact, but more likely is approximated via
nearest neighbor descent.
Parameters
----------
X: array of shape (n_samples, n_features)
The input data to compute the k-neighbor graph of.
n_neighbors: int
The number of nearest neighbors to compute for each sample in ``X``.
metric: string or callable
The metric to use for the computation.
metric_kwds: dict
Any arguments to pass to the metric computation function.
angular: bool
Whether to use angular rp trees in NN approximation.
random_state: np.random state
The random state to use for approximate NN computations.
low_memory: bool (optional, default True)
Whether to pursue lower memory NNdescent.
verbose: bool (optional, default False)
Whether to print status data during the computation.
Returns
-------
knn_indices: array of shape (n_samples, n_neighbors)
The indices on the ``n_neighbors`` closest points in the dataset.
knn_dists: array of shape (n_samples, n_neighbors)
The distances to the ``n_neighbors`` closest points in the dataset.
rp_forest: list of trees
The random projection forest used for searching (if used, None otherwise).
"""
if verbose:
print(ts(), "Finding Nearest Neighbors")
if metric == "precomputed":
# Note that this does not support sparse distance matrices yet ...
# Compute indices of n nearest neighbors
knn_indices = fast_knn_indices(X, n_neighbors)
# knn_indices = np.argsort(X)[:, :n_neighbors]
# Compute the nearest neighbor distances
# (equivalent to np.sort(X)[:,:n_neighbors])
# Advanced indexing already returns a fresh contiguous array, so no
# extra .copy() is needed here.
knn_dists = X[np.arange(X.shape[0])[:, None], knn_indices]
# Prune any nearest neighbours that are infinite distance apart.
disconnected_index = knn_dists == np.inf
knn_indices[disconnected_index] = -1
knn_search_index = None
else:
# TODO: Hacked values for now
n_trees = min(64, 5 + int(round((X.shape[0]) ** 0.5 / 20.0)))
n_iters = max(5, int(round(np.log2(X.shape[0]))))
knn_search_index = NNDescent(
X,
n_neighbors=n_neighbors,
metric=metric,
metric_kwds=metric_kwds,
random_state=random_state,
n_trees=n_trees,
n_iters=n_iters,
max_candidates=60,
low_memory=low_memory,
n_jobs=n_jobs,
verbose=verbose,
compressed=False,
)
knn_indices, knn_dists = knn_search_index.neighbor_graph
if verbose:
print(ts(), "Finished Nearest Neighbor Search")
return knn_indices, knn_dists, knn_search_index
@numba.njit(
locals={
"knn_dists": numba.types.float32[:, ::1],
"sigmas": numba.types.float32[::1],
"rhos": numba.types.float32[::1],
"val": numba.types.float32,
},
parallel=True,
)
def compute_membership_strengths(
knn_indices,
knn_dists,
sigmas,
rhos,
return_dists=False,
bipartite=False,
):
"""Construct the membership strength data for the 1-skeleton of each local
fuzzy simplicial set -- this is formed as a sparse matrix where each row is
a local fuzzy simplicial set, with a membership strength for the
1-simplex to each other data point.
Parameters
----------
knn_indices: array of shape (n_samples, n_neighbors)
The indices on the ``n_neighbors`` closest points in the dataset.
knn_dists: array of shape (n_samples, n_neighbors)
The distances to the ``n_neighbors`` closest points in the dataset.
sigmas: array of shape(n_samples)
The normalization factor derived from the metric tensor approximation.
rhos: array of shape(n_samples)
The local connectivity adjustment.
return_dists: bool (optional, default False)
Whether to return the pairwise distance associated with each edge.
bipartite: bool (optional, default False)
Does the nearest neighbour set represent a bipartite graph? That is, are the
nearest neighbour indices from the same point set as the row indices?
Returns
-------
rows: array of shape (n_samples * n_neighbors)
Row data for the resulting sparse matrix (coo format)
cols: array of shape (n_samples * n_neighbors)
Column data for the resulting sparse matrix (coo format)
vals: array of shape (n_samples * n_neighbors)
Entries for the resulting sparse matrix (coo format)
dists: array of shape (n_samples * n_neighbors)
Distance associated with each entry in the resulting sparse matrix
"""
n_samples = knn_indices.shape[0]
n_neighbors = knn_indices.shape[1]
rows = np.zeros(knn_indices.size, dtype=np.int32)
cols = np.zeros(knn_indices.size, dtype=np.int32)
vals = np.zeros(knn_indices.size, dtype=np.float32)
if return_dists:
dists = np.zeros(knn_indices.size, dtype=np.float32)
else:
dists = None
for i in range(n_samples):
for j in range(n_neighbors):
if knn_indices[i, j] == -1:
continue # We didn't get the full knn for i
# If applied to an adjacency matrix points shouldn't be similar to themselves.
# If applied to an incidence matrix (or bipartite) then the row and column indices are different.
if (bipartite == False) & (knn_indices[i, j] == i):
val = 0.0
elif knn_dists[i, j] - rhos[i] <= 0.0 or sigmas[i] == 0.0:
val = 1.0
else:
val = np.exp(-((knn_dists[i, j] - rhos[i]) / (sigmas[i])))
rows[i * n_neighbors + j] = i
cols[i * n_neighbors + j] = knn_indices[i, j]
vals[i * n_neighbors + j] = val
if return_dists:
dists[i * n_neighbors + j] = knn_dists[i, j]
return rows, cols, vals, dists
def fuzzy_simplicial_set(
X,
n_neighbors,
random_state,
metric,
metric_kwds={},
knn_indices=None,
knn_dists=None,
angular=False,
set_op_mix_ratio=1.0,
local_connectivity=1.0,
apply_set_operations=True,
verbose=False,
return_dists=None,
):
"""Given a set of data X, a neighborhood size, and a measure of distance
compute the fuzzy simplicial set (here represented as a fuzzy graph in
the form of a sparse matrix) associated to the data. This is done by
locally approximating geodesic distance at each point, creating a fuzzy
simplicial set for each such point, and then combining all the local
fuzzy simplicial sets into a global one via a fuzzy union.
Parameters
----------
X: array of shape (n_samples, n_features)
The data to be modelled as a fuzzy simplicial set.
n_neighbors: int
The number of neighbors to use to approximate geodesic distance.
Larger numbers induce more global estimates of the manifold that can
miss finer detail, while smaller values will focus on fine manifold
structure to the detriment of the larger picture.
random_state: numpy RandomState or equivalent
A state capable being used as a numpy random state.
metric: string or function (optional, default 'euclidean')
The metric to use to compute distances in high dimensional space.
If a string is passed it must match a valid predefined metric. If
a general metric is required a function that takes two 1d arrays and
returns a float can be provided. For performance purposes it is
required that this be a numba jit'd function. Valid string metrics
include:
* euclidean (or l2)
* manhattan (or l1)
* cityblock
* braycurtis
* canberra
* chebyshev
* correlation
* cosine
* dice
* hamming
* jaccard
* kulsinski
* ll_dirichlet
* mahalanobis
* matching
* minkowski
* rogerstanimoto
* russellrao
* seuclidean
* sokalmichener
* sokalsneath
* sqeuclidean
* yule
* wminkowski
Metrics that take arguments (such as minkowski, mahalanobis etc.)
can have arguments passed via the metric_kwds dictionary. At this
time care must be taken and dictionary elements must be ordered
appropriately; this will hopefully be fixed in the future.
metric_kwds: dict (optional, default {})
Arguments to pass on to the metric, such as the ``p`` value for
Minkowski distance.
knn_indices: array of shape (n_samples, n_neighbors) (optional)
If the k-nearest neighbors of each point has already been calculated
you can pass them in here to save computation time. This should be
an array with the indices of the k-nearest neighbors as a row for
each data point.
knn_dists: array of shape (n_samples, n_neighbors) (optional)
If the k-nearest neighbors of each point has already been calculated
you can pass them in here to save computation time. This should be
an array with the distances of the k-nearest neighbors as a row for
each data point.
angular: bool (optional, default False)
Whether to use angular/cosine distance for the random projection
forest for seeding NN-descent to determine approximate nearest
neighbors.
set_op_mix_ratio: float (optional, default 1.0)
Interpolate between (fuzzy) union and intersection as the set operation
used to combine local fuzzy simplicial sets to obtain a global fuzzy
simplicial sets. Both fuzzy set operations use the product t-norm.
The value of this parameter should be between 0.0 and 1.0; a value of
1.0 will use a pure fuzzy union, while 0.0 will use a pure fuzzy
intersection.
local_connectivity: int (optional, default 1)
The local connectivity required -- i.e. the number of nearest
neighbors that should be assumed to be connected at a local level.
The higher this value the more connected the manifold becomes
locally. In practice this should be not more than the local intrinsic
dimension of the manifold.
verbose: bool (optional, default False)
Whether to report information on the current progress of the algorithm.
return_dists: bool or None (optional, default None)
Whether to return the pairwise distance associated with each edge.
Returns
-------
fuzzy_simplicial_set: coo_matrix
A fuzzy simplicial set represented as a sparse matrix. The (i,
j) entry of the matrix represents the membership strength of the
1-simplex between the ith and jth sample points.
"""
if knn_indices is None or knn_dists is None:
knn_indices, knn_dists, _ = nearest_neighbors(
X,
n_neighbors,
metric,
metric_kwds,
angular,
random_state,
verbose=verbose,
)
knn_dists = knn_dists.astype(np.float32, copy=False)
sigmas, rhos = smooth_knn_dist(
knn_dists,
float(n_neighbors),
local_connectivity=float(local_connectivity),
)
rows, cols, vals, dists = compute_membership_strengths(
knn_indices, knn_dists, sigmas, rhos, return_dists
)
result = scipy.sparse.coo_matrix(
(vals, (rows, cols)), shape=(X.shape[0], X.shape[0])
)
result.eliminate_zeros()
if apply_set_operations:
transpose = result.transpose()
prod_matrix = result.multiply(transpose)
if set_op_mix_ratio == 1.0:
# Default fuzzy union: the (1 - ratio) * prod_matrix term is zero, so
# skip building/scaling/adding it (saves redundant sparse temporaries).
result = result + transpose - prod_matrix
elif set_op_mix_ratio == 0.0:
# Pure fuzzy intersection.
result = prod_matrix
else:
result = (
set_op_mix_ratio * (result + transpose - prod_matrix)
+ (1.0 - set_op_mix_ratio) * prod_matrix
)
result.eliminate_zeros()
if return_dists is None:
return result, sigmas, rhos
else:
if return_dists:
dmat = scipy.sparse.coo_matrix(
(dists, (rows, cols)), shape=(X.shape[0], X.shape[0])
)
dists = dmat.maximum(dmat.transpose()).todok()
else:
dists = None
return result, sigmas, rhos, dists
@numba.njit()
def fast_intersection(rows, cols, values, target, unknown_dist=1.0, far_dist=5.0):
"""Under the assumption of categorical distance for the intersecting
simplicial set perform a fast intersection.
Parameters
----------
rows: array
An array of the row of each non-zero in the sparse matrix
representation.
cols: array
An array of the column of each non-zero in the sparse matrix
representation.
values: array
An array of the value of each non-zero in the sparse matrix
representation.
target: array of shape (n_samples)
The categorical labels to use in the intersection.
unknown_dist: float (optional, default 1.0)
The distance an unknown label (-1) is assumed to be from any point.
far_dist float (optional, default 5.0)
The distance between unmatched labels.
Returns
-------
None
"""
for nz in range(rows.shape[0]):
i = rows[nz]
j = cols[nz]
if (target[i] == -1) or (target[j] == -1):
values[nz] *= np.exp(-unknown_dist)
elif target[i] != target[j]:
values[nz] *= np.exp(-far_dist)
return
@numba.njit()
def fast_metric_intersection(
rows, cols, values, discrete_space, metric, metric_args, scale
):
"""Under the assumption of categorical distance for the intersecting
simplicial set perform a fast intersection.
Parameters
----------
rows: array
An array of the row of each non-zero in the sparse matrix
representation.
cols: array
An array of the column of each non-zero in the sparse matrix
representation.
values: array of shape
An array of the values of each non-zero in the sparse matrix
representation.
discrete_space: array of shape (n_samples, n_features)
The vectors of categorical labels to use in the intersection.
metric: numba function
The function used to calculate distance over the target array.
scale: float
A scaling to apply to the metric.
Returns
-------
None
"""
for nz in range(rows.shape[0]):
i = rows[nz]
j = cols[nz]
dist = metric(discrete_space[i], discrete_space[j], *metric_args)
values[nz] *= np.exp(-(scale * dist))
return
@numba.njit()
def reprocess_row(probabilities, k=15, n_iters=32):
target = np.log2(k)
lo = 0.0
hi = NPY_INFINITY
mid = 1.0
for n in range(n_iters):
psum = 0.0
for j in range(probabilities.shape[0]):
psum += pow(probabilities[j], mid)
if np.fabs(psum - target) < SMOOTH_K_TOLERANCE:
break
if psum < target:
hi = mid
mid = (lo + hi) / 2.0
else:
lo = mid
if hi == NPY_INFINITY:
mid *= 2
else:
mid = (lo + hi) / 2.0
return np.power(probabilities, mid)
@numba.njit()
def reset_local_metrics(simplicial_set_indptr, simplicial_set_data):
for i in range(simplicial_set_indptr.shape[0] - 1):
simplicial_set_data[simplicial_set_indptr[i] : simplicial_set_indptr[i + 1]] = (
reprocess_row(
simplicial_set_data[
simplicial_set_indptr[i] : simplicial_set_indptr[i + 1]
]
)
)
return
def reset_local_connectivity(simplicial_set, reset_local_metric=False):
"""Reset the local connectivity requirement -- each data sample should
have complete confidence in at least one 1-simplex in the simplicial set.
We can enforce this by locally rescaling confidences, and then remerging the
different local simplicial sets together.
Parameters
----------
simplicial_set: sparse matrix
The simplicial set for which to recalculate with respect to local
connectivity.
Returns
-------
simplicial_set: sparse_matrix
The recalculated simplicial set, now with the local connectivity
assumption restored.
"""
simplicial_set = normalize(simplicial_set, norm="max")
if reset_local_metric:
simplicial_set = simplicial_set.tocsr()
reset_local_metrics(simplicial_set.indptr, simplicial_set.data)
simplicial_set = simplicial_set.tocoo()
transpose = simplicial_set.transpose()
prod_matrix = simplicial_set.multiply(transpose)
simplicial_set = simplicial_set + transpose - prod_matrix
simplicial_set.eliminate_zeros()
return simplicial_set
def discrete_metric_simplicial_set_intersection(
simplicial_set,
discrete_space,
unknown_dist=1.0,
far_dist=5.0,
metric=None,
metric_kws={},
metric_scale=1.0,
):
"""Combine a fuzzy simplicial set with another fuzzy simplicial set
generated from discrete metric data using discrete distances. The target
data is assumed to be categorical label data (a vector of labels),
and this will update the fuzzy simplicial set to respect that label data.
TODO: optional category cardinality based weighting of distance
Parameters
----------
simplicial_set: sparse matrix
The input fuzzy simplicial set.
discrete_space: array of shape (n_samples)
The categorical labels to use in the intersection.
unknown_dist: float (optional, default 1.0)
The distance an unknown label (-1) is assumed to be from any point.
far_dist: float (optional, default 5.0)
The distance between unmatched labels.
metric: str (optional, default None)
If not None, then use this metric to determine the
distance between values.
metric_scale: float (optional, default 1.0)
If using a custom metric scale the distance values by
this value -- this controls the weighting of the
intersection. Larger values weight more toward target.
Returns
-------
simplicial_set: sparse matrix
The resulting intersected fuzzy simplicial set.
"""
simplicial_set = simplicial_set.tocoo()
if metric is not None:
# We presume target is now a 2d array, with each row being a
# vector of target info
if metric in dist.named_distances:
metric_func = dist.named_distances[metric]
else:
raise ValueError("Discrete intersection metric is not recognized")
fast_metric_intersection(
simplicial_set.row,
simplicial_set.col,
simplicial_set.data,
discrete_space,
metric_func,
tuple(metric_kws.values()),
metric_scale,
)
else:
fast_intersection(
simplicial_set.row,
simplicial_set.col,
simplicial_set.data,
discrete_space,
unknown_dist,
far_dist,
)
simplicial_set.eliminate_zeros()
return reset_local_connectivity(simplicial_set)
def general_simplicial_set_intersection(
simplicial_set1, simplicial_set2, weight=0.5, right_complement=False
):
if right_complement:
result = simplicial_set1.tocoo()
else:
result = (simplicial_set1 + simplicial_set2).tocoo()
left = simplicial_set1.tocsr()
right = simplicial_set2.tocsr()
sparse.general_sset_intersection(
left.indptr,
left.indices,
left.data,
right.indptr,
right.indices,
right.data,
result.row,
result.col,
result.data,
mix_weight=weight,
right_complement=right_complement,
)
return result
def general_simplicial_set_union(simplicial_set1, simplicial_set2):
result = (simplicial_set1 + simplicial_set2).tocoo()
left = simplicial_set1.tocsr()
right = simplicial_set2.tocsr()
sparse.general_sset_union(
left.indptr,
left.indices,
left.data,
right.indptr,
right.indices,
right.data,
result.row,
result.col,
result.data,
)
return result
def make_epochs_per_sample(weights, n_epochs):
"""Given a set of weights and number of epochs generate the number of
epochs per sample for each weight.
Parameters
----------
weights: array of shape (n_1_simplices)
The weights of how much we wish to sample each 1-simplex.
n_epochs: int
The total number of epochs we want to train for.
Returns
-------
An array of number of epochs per sample, one for each 1-simplex.
"""
result = np.full(weights.shape[0], -1.0, dtype=np.float64)
n_samples = n_epochs * (weights / weights.max())
positive = n_samples > 0 # compute the mask once instead of twice
result[positive] = float(n_epochs) / np.float64(n_samples[positive])
return result
# scale coords so that the largest coordinate is max_coords, then add normal-distributed
# noise with standard deviation noise
def noisy_scale_coords(coords, random_state, max_coord=10.0, noise=0.0001):
expansion = max_coord / np.abs(coords).max()
coords = (coords * expansion).astype(np.float32)
return coords + random_state.normal(scale=noise, size=coords.shape).astype(
np.float32
)
@numba.njit()
def _densmap_original_densities(
head, tail, graph_data, dists_indptr, dists_indices, dists_data, ro, mu_sum
):
for i in range(len(head)):
j = head[i]
k = tail[i]
d_val = 0.0
for idx in range(dists_indptr[j], dists_indptr[j + 1]):
if dists_indices[idx] == k:
d_val = dists_data[idx]
break
D = d_val * d_val
mu = graph_data[i]
ro[j] += mu * D
ro[k] += mu * D
mu_sum[j] += mu
mu_sum[k] += mu
@numba.njit()
def _densmap_embedding_densities(
head, tail, graph_data, dists_indptr, dists_indices, dists_data, re, mu_sum
):
for i in range(len(head)):
j = head[i]
k = tail[i]
d_val = 0.0
for idx in range(dists_indptr[j], dists_indptr[j + 1]):
if dists_indices[idx] == k:
d_val = dists_data[idx]
break
mu = graph_data[i]
weighted = mu * d_val
re[j] += weighted
re[k] += weighted
mu_sum[j] += mu
mu_sum[k] += mu
def simplicial_set_embedding(
data,
graph,
n_components,
initial_alpha,
a,
b,
gamma,
negative_sample_rate,
n_epochs,
init,
random_state,
metric,
metric_kwds,
densmap,
densmap_kwds,
output_dens,
output_metric=dist.named_distances_with_gradients["euclidean"],
output_metric_kwds={},
euclidean_output=True,
parallel=False,
verbose=False,
tqdm_kwds=None,
):
"""Perform a fuzzy simplicial set embedding, using a specified
initialisation method and then minimizing the fuzzy set cross entropy
between the 1-skeletons of the high and low dimensional fuzzy simplicial
sets.
Parameters
----------
data: array of shape (n_samples, n_features)
The source data to be embedded by UMAP.
graph: sparse matrix
The 1-skeleton of the high dimensional fuzzy simplicial set as
represented by a graph for which we require a sparse matrix for the
(weighted) adjacency matrix.
n_components: int
The dimensionality of the euclidean space into which to embed the data.
initial_alpha: float
Initial learning rate for the SGD.
a: float
Parameter of differentiable approximation of right adjoint functor
b: float
Parameter of differentiable approximation of right adjoint functor
gamma: float
Weight to apply to negative samples.
negative_sample_rate: int (optional, default 5)
The number of negative samples to select per positive sample
in the optimization process. Increasing this value will result
in greater repulsive force being applied, greater optimization
cost, but slightly more accuracy.
n_epochs: int (optional, default 0), or list of int
The number of training epochs to be used in optimizing the
low dimensional embedding. Larger values result in more accurate
embeddings. If 0 is specified a value will be selected based on
the size of the input dataset (200 for large datasets, 500 for small).
If a list of int is specified, then the intermediate embeddings at the
different epochs specified in that list are returned in
``aux_data["embedding_list"]``.
init: string
How to initialize the low dimensional embedding. Options are:
* 'spectral': use a spectral embedding of the fuzzy 1-skeleton
* 'random': assign initial embedding positions at random.
* 'pca': use the first n_components from PCA applied to the input data.
* A numpy array of initial embedding positions.
random_state: numpy RandomState or equivalent
A state capable being used as a numpy random state.
metric: string or callable
The metric used to measure distance in high dimensional space; used if
multiple connected components need to be layed out.
metric_kwds: dict
Key word arguments to be passed to the metric function; used if
multiple connected components need to be layed out.
densmap: bool
Whether to use the density-augmented objective function to optimize
the embedding according to the densMAP algorithm.
densmap_kwds: dict
Key word arguments to be used by the densMAP optimization.
output_dens: bool
Whether to output local radii in the original data and the embedding.
output_metric: function
Function returning the distance between two points in embedding space and
the gradient of the distance wrt the first argument.
output_metric_kwds: dict
Key word arguments to be passed to the output_metric function.
euclidean_output: bool
Whether to use the faster code specialised for euclidean output metrics
parallel: bool (optional, default False)
Whether to run the computation using numba parallel.
Running in parallel is non-deterministic, and is not used
if a random seed has been set, to ensure reproducibility.
verbose: bool (optional, default False)
Whether to report information on the current progress of the algorithm.
tqdm_kwds: dict
Key word arguments to be used by the tqdm progress bar.
Returns
-------
embedding: array of shape (n_samples, n_components)
The optimized of ``graph`` into an ``n_components`` dimensional
euclidean space.
aux_data: dict
Auxiliary output returned with the embedding. When densMAP extension
is turned on, this dictionary includes local radii in the original
data (``rad_orig``) and in the embedding (``rad_emb``).
"""
graph = graph.tocoo()
graph.sum_duplicates()
n_vertices = graph.shape[1]
# For smaller datasets we can use more epochs
if graph.shape[0] <= 10000:
default_epochs = 500
else:
default_epochs = 200
# Use more epochs for densMAP
if densmap:
default_epochs += 200
if n_epochs is None:
n_epochs = default_epochs
# If n_epoch is a list, get the maximum epoch to reach
n_epochs_max = max(n_epochs) if isinstance(n_epochs, list) else n_epochs
if n_epochs_max > 10:
graph.data[graph.data < (graph.data.max() / float(n_epochs_max))] = 0.0
else:
graph.data[graph.data < (graph.data.max() / float(default_epochs))] = 0.0
graph.eliminate_zeros()
if isinstance(init, str) and init == "random":
embedding = random_state.uniform(
low=-10.0, high=10.0, size=(graph.shape[0], n_components)
).astype(np.float32)
elif isinstance(init, str) and init == "pca":
if scipy.sparse.issparse(data):
pca = TruncatedSVD(n_components=n_components, random_state=random_state)
else:
pca = PCA(n_components=n_components, random_state=random_state)
embedding = pca.fit_transform(data).astype(np.float32)
embedding = noisy_scale_coords(
embedding, random_state, max_coord=10, noise=0.0001
)
elif isinstance(init, str) and init == "spectral":
embedding = spectral_layout(
data,
graph,
n_components,
random_state,
metric=metric,
metric_kwds=metric_kwds,
)
# We add a little noise to avoid local minima for optimization to come
embedding = noisy_scale_coords(
embedding, random_state, max_coord=10, noise=0.0001
)
elif isinstance(init, str) and init == "tswspectral":
embedding = tswspectral_layout(
data,
graph,
n_components,
random_state,
metric=metric,
metric_kwds=metric_kwds,
)
embedding = noisy_scale_coords(
embedding, random_state, max_coord=10, noise=0.0001
)
else:
init_data = np.array(init)
if len(init_data.shape) == 2:
if np.unique(init_data, axis=0).shape[0] < init_data.shape[0]:
tree = KDTree(init_data)
dist, ind = tree.query(init_data, k=2)
nndist = np.mean(dist[:, 1])
embedding = init_data + random_state.normal(
scale=0.001 * nndist, size=init_data.shape
).astype(np.float32)
else:
embedding = init_data
epochs_per_sample = make_epochs_per_sample(graph.data, n_epochs_max)
head = graph.row
tail = graph.col
weight = graph.data
rng_state = random_state.randint(INT32_MIN, INT32_MAX, 3).astype(np.int64)
aux_data = {}
if densmap or output_dens:
if verbose:
print(ts() + " Computing original densities")
dists = densmap_kwds["graph_dists"]
mu_sum = np.zeros(n_vertices, dtype=np.float32)
ro = np.zeros(n_vertices, dtype=np.float32)
dists_csr = dists.tocsr()
_densmap_original_densities(
head,
tail,
graph.data,
dists_csr.indptr,
dists_csr.indices,
dists_csr.data,
ro,
mu_sum,
)
epsilon = 1e-8
ro = np.log(epsilon + (ro / mu_sum))
if densmap:
R = (ro - np.mean(ro)) / np.std(ro)
densmap_kwds["mu"] = graph.data
densmap_kwds["mu_sum"] = mu_sum
densmap_kwds["R"] = R
if output_dens:
aux_data["rad_orig"] = ro
embedding = (
10.0
* (embedding - np.min(embedding, 0))
/ (np.max(embedding, 0) - np.min(embedding, 0))
).astype(np.float32, order="C")
if euclidean_output:
embedding = optimize_layout_euclidean(
embedding,
embedding,
head,
tail,
n_epochs,
n_vertices,
epochs_per_sample,
a,
b,
rng_state,
gamma,
initial_alpha,
negative_sample_rate,
parallel=parallel,
verbose=verbose,
densmap=densmap,
densmap_kwds=densmap_kwds,
tqdm_kwds=tqdm_kwds,
move_other=True,
)
else:
embedding = optimize_layout_generic(
embedding,
embedding,
head,
tail,
n_epochs,
n_vertices,
epochs_per_sample,
a,
b,
rng_state,
gamma,
initial_alpha,
negative_sample_rate,
output_metric,
tuple(output_metric_kwds.values()),
verbose=verbose,
tqdm_kwds=tqdm_kwds,
move_other=True,
)
if isinstance(embedding, list):
aux_data["embedding_list"] = embedding
embedding = embedding[-1].copy()
if output_dens:
if verbose:
print(ts() + " Computing embedding densities")
# Compute graph in embedding
(
knn_indices,
knn_dists,
rp_forest,
) = nearest_neighbors(
embedding,
densmap_kwds["n_neighbors"],
"euclidean",
{},
False,
random_state,
verbose=verbose,
)
emb_graph, emb_sigmas, emb_rhos, emb_dists = fuzzy_simplicial_set(
embedding,
densmap_kwds["n_neighbors"],
random_state,
"euclidean",
{},
knn_indices,
knn_dists,
verbose=verbose,
return_dists=True,
)
emb_graph = emb_graph.tocoo()
emb_graph.sum_duplicates()
emb_graph.eliminate_zeros()
n_vertices = emb_graph.shape[1]
mu_sum = np.zeros(n_vertices, dtype=np.float32)
re = np.zeros(n_vertices, dtype=np.float32)
head = emb_graph.row
tail = emb_graph.col
emb_dists_csr = emb_dists.tocsr()
_densmap_embedding_densities(
head,
tail,
emb_graph.data,
emb_dists_csr.indptr,
emb_dists_csr.indices,
emb_dists_csr.data,
re,
mu_sum,
)
epsilon = 1e-8
re = np.log(epsilon + (re / mu_sum))
aux_data["rad_emb"] = re
return embedding, aux_data
@numba.njit()
def init_transform(indices, weights, embedding):
"""Given indices and weights and an original embeddings
initialize the positions of new points relative to the
indices and weights (of their neighbors in the source data).
Parameters
----------
indices: array of shape (n_new_samples, n_neighbors)
The indices of the neighbors of each new sample
weights: array of shape (n_new_samples, n_neighbors)
The membership strengths of associated 1-simplices
for each of the new samples.
embedding: array of shape (n_samples, dim)
The original embedding of the source data.
Returns
-------
new_embedding: array of shape (n_new_samples, dim)
An initial embedding of the new sample points.
"""
result = np.zeros((indices.shape[0], embedding.shape[1]), dtype=np.float32)
for i in range(indices.shape[0]):
for j in range(indices.shape[1]):
for d in range(embedding.shape[1]):
result[i, d] += weights[i, j] * embedding[indices[i, j], d]
return result
def init_graph_transform(graph, embedding):
"""Given a bipartite graph representing the 1-simplices and strengths between the
new points and the original data set along with an embedding of the original points
initialize the positions of new points relative to the strengths (of their neighbors in the source data).
If a point is in our original data set it embeds at the original points coordinates.
If a point has no neighbours in our original dataset it embeds as the np.nan vector.
Otherwise a point is the weighted average of it's neighbours embedding locations.
Parameters
----------
graph: csr_matrix (n_new_samples, n_samples)
A matrix indicating the 1-simplices and their associated strengths. These strengths should
be values between zero and one and not normalized. One indicating that the new point was identical
to one of our original points.
embedding: array of shape (n_samples, dim)
The original embedding of the source data.
Returns
-------
new_embedding: array of shape (n_new_samples, dim)
An initial embedding of the new sample points.
"""
n_new = graph.shape[0]
result = np.zeros((n_new, embedding.shape[1]), dtype=np.float32)
row_nnz = np.diff(graph.indptr)
empty_mask = row_nnz == 0
result[empty_mask] = np.nan
has_exact = np.zeros(n_new, dtype=bool)
exact_data_mask = graph.data == 1.0
if exact_data_mask.any():
exact_positions = np.where(exact_data_mask)[0]
exact_rows = np.searchsorted(graph.indptr, exact_positions, side="right") - 1
_, first_idx = np.unique(exact_rows, return_index=True)
unique_rows = exact_rows[first_idx]
exact_cols = graph.indices[exact_positions[first_idx]]
has_exact[unique_rows] = True
result[unique_rows] = embedding[exact_cols]
avg_mask = ~empty_mask & ~has_exact
if np.any(avg_mask):
avg_graph = graph[avg_mask]
row_sums = np.array(avg_graph.sum(axis=1)).flatten()
inv_sums = scipy.sparse.diags(1.0 / row_sums)
normalized = inv_sums @ avg_graph
result[avg_mask] = (normalized @ embedding).astype(np.float32)
return result
@numba.njit()
def init_update(current_init, n_original_samples, indices):
# Points without any original neighbours start at the centre of the
# original embedding
centre = np.zeros(current_init.shape[1], dtype=np.float64)
for k in range(n_original_samples):
for d in range(current_init.shape[1]):
centre[d] += current_init[k, d]
centre /= n_original_samples
for i in range(n_original_samples, indices.shape[0]):
n = 0
for j in range(indices.shape[1]):
if indices[i, j] < n_original_samples:
n += 1
for d in range(current_init.shape[1]):
current_init[i, d] += current_init[indices[i, j], d]
for d in range(current_init.shape[1]):
if n > 0:
current_init[i, d] /= n
else:
current_init[i, d] = centre[d]
return
def find_ab_params(spread, min_dist):
"""Fit a, b params for the differentiable curve used in lower
dimensional fuzzy simplicial complex construction. We want the
smooth curve (from a pre-defined family with simple gradient) that
best matches an offset exponential decay.
"""
def curve(x, a, b):
return 1.0 / (1.0 + a * x ** (2 * b))
xv = np.linspace(0, spread * 3, 300)
yv = np.zeros(xv.shape)
yv[xv < min_dist] = 1.0
yv[xv >= min_dist] = np.exp(-(xv[xv >= min_dist] - min_dist) / spread)
params, covar = curve_fit(curve, xv, yv)
return params[0], params[1]
class UMAP(BaseEstimator, ClassNamePrefixFeaturesOutMixin):
"""Uniform Manifold Approximation and Projection
Finds a low dimensional embedding of the data that approximates
an underlying manifold.
Parameters
----------
n_neighbors: float (optional, default 15)
The size of local neighborhood (in terms of number of neighboring
sample points) used for manifold approximation. Larger values
result in more global views of the manifold, while smaller
values result in more local data being preserved. In general
values should be in the range 2 to 100.
n_components: int (optional, default 2)
The dimension of the space to embed into. This defaults to 2 to
provide easy visualization, but can reasonably be set to any
integer value in the range 2 to 100.
metric: string or function (optional, default 'euclidean')
The metric to use to compute distances in high dimensional space.
If a string is passed it must match a valid predefined metric. If
a general metric is required a function that takes two 1d arrays and
returns a float can be provided. For performance purposes it is
required that this be a numba jit'd function. Valid string metrics
include:
* euclidean
* manhattan
* chebyshev
* minkowski
* canberra
* braycurtis
* mahalanobis
* wminkowski
* seuclidean
* cosine
* correlation
* haversine
* hamming
* jaccard
* dice
* russelrao
* kulsinski
* ll_dirichlet
* hellinger
* rogerstanimoto
* sokalmichener
* sokalsneath
* yule
Metrics that take arguments (such as minkowski, mahalanobis etc.)
can have arguments passed via the metric_kwds dictionary. At this
time care must be taken and dictionary elements must be ordered
appropriately; this will hopefully be fixed in the future.
n_epochs: int (optional, default None)
The number of training epochs to be used in optimizing the
low dimensional embedding. Larger values result in more accurate
embeddings. If None is specified a value will be selected based on
the size of the input dataset (200 for large datasets, 500 for small).
learning_rate: float (optional, default 1.0)
The initial learning rate for the embedding optimization.
init: string (optional, default 'spectral')
How to initialize the low dimensional embedding. Options are:
* 'spectral': use a spectral embedding of the fuzzy 1-skeleton
* 'random': assign initial embedding positions at random.
* 'pca': use the first n_components from PCA applied to the
input data.
* 'tswspectral': use a spectral embedding of the fuzzy
1-skeleton, using a truncated singular value decomposition to
"warm" up the eigensolver. This is intended as an alternative
to the 'spectral' method, if that takes an excessively long
time to complete initialization (or fails to complete).
* A numpy array of initial embedding positions.
min_dist: float (optional, default 0.1)
The effective minimum distance between embedded points. Smaller values
will result in a more clustered/clumped embedding where nearby points
on the manifold are drawn closer together, while larger values will
result on a more even dispersal of points. The value should be set
relative to the ``spread`` value, which determines the scale at which
embedded points will be spread out.
spread: float (optional, default 1.0)
The effective scale of embedded points. In combination with ``min_dist``
this determines how clustered/clumped the embedded points are.
low_memory: bool (optional, default True)
For some datasets the nearest neighbor computation can consume a lot of
memory. If you find that UMAP is failing due to memory constraints
consider setting this option to True. This approach is more
computationally expensive, but avoids excessive memory use.
set_op_mix_ratio: float (optional, default 1.0)
Interpolate between (fuzzy) union and intersection as the set operation
used to combine local fuzzy simplicial sets to obtain a global fuzzy
simplicial sets. Both fuzzy set operations use the product t-norm.
The value of this parameter should be between 0.0 and 1.0; a value of
1.0 will use a pure fuzzy union, while 0.0 will use a pure fuzzy
intersection.
local_connectivity: int (optional, default 1)
The local connectivity required -- i.e. the number of nearest
neighbors that should be assumed to be connected at a local level.
The higher this value the more connected the manifold becomes
locally. In practice this should be not more than the local intrinsic
dimension of the manifold.
repulsion_strength: float (optional, default 1.0)
Weighting applied to negative samples in low dimensional embedding
optimization. Values higher than one will result in greater weight
being given to negative samples.
negative_sample_rate: int (optional, default 5)
The number of negative samples to select per positive sample
in the optimization process. Increasing this value will result
in greater repulsive force being applied, greater optimization
cost, but slightly more accuracy.
transform_queue_size: float (optional, default 4.0)
For transform operations (embedding new points using a trained model
this will control how aggressively to search for nearest neighbors.
Larger values will result in slower performance but more accurate
nearest neighbor evaluation.
a: float (optional, default None)
More specific parameters controlling the embedding. If None these
values are set automatically as determined by ``min_dist`` and
``spread``.
b: float (optional, default None)
More specific parameters controlling the embedding. If None these
values are set automatically as determined by ``min_dist`` and
``spread``.
random_state: int, RandomState instance or None, optional (default: None)
If int, random_state is the seed used by the random number generator;
If RandomState instance, random_state is the random number generator;
If None, the random number generator is the RandomState instance used
by `np.random`.
metric_kwds: dict (optional, default None)
Arguments to pass on to the metric, such as the ``p`` value for
Minkowski distance. If None then no arguments are passed on.
angular_rp_forest: bool (optional, default False)
Whether to use an angular random projection forest to initialise
the approximate nearest neighbor search. This can be faster, but is
mostly only useful for a metric that uses an angular style distance such
as cosine, correlation etc. In the case of those metrics angular forests
will be chosen automatically.
target_n_neighbors: int (optional, default -1)
The number of nearest neighbors to use to construct the target simplicial
set. If set to -1 use the ``n_neighbors`` value.
target_metric: string or callable (optional, default 'categorical')
The metric used to measure distance for a target array is using supervised
dimension reduction. By default this is 'categorical' which will measure
distance in terms of whether categories match or are different. Furthermore,
if semi-supervised is required target values of -1 will be trated as
unlabelled under the 'categorical' metric. If the target array takes
continuous values (e.g. for a regression problem) then metric of 'l1'
or 'l2' is probably more appropriate.
target_metric_kwds: dict (optional, default None)
Keyword argument to pass to the target metric when performing
supervised dimension reduction. If None then no arguments are passed on.
target_weight: float (optional, default 0.5)
weighting factor between data topology and target topology. A value of
0.0 weights predominantly on data, a value of 1.0 places a strong emphasis on
target. The default of 0.5 balances the weighting equally between data and
target.
transform_seed: int (optional, default 42)
Random seed used for the stochastic aspects of the transform operation.
This ensures consistency in transform operations.
verbose: bool (optional, default False)
Controls verbosity of logging.
tqdm_kwds: dict (optional, defaul None)
Key word arguments to be used by the tqdm progress bar.
unique: bool (optional, default False)
Controls if the rows of your data should be uniqued before being
embedded. If you have more duplicates than you have ``n_neighbors``
you can have the identical data points lying in different regions of
your space. It also violates the definition of a metric.
For to map from internal structures back to your data use the variable
_unique_inverse_.
densmap: bool (optional, default False)
Specifies whether the density-augmented objective of densMAP
should be used for optimization. Turning on this option generates
an embedding where the local densities are encouraged to be correlated
with those in the original space. Parameters below with the prefix 'dens'
further control the behavior of this extension.
dens_lambda: float (optional, default 2.0)
Controls the regularization weight of the density correlation term
in densMAP. Higher values prioritize density preservation over the
UMAP objective, and vice versa for values closer to zero. Setting this
parameter to zero is equivalent to running the original UMAP algorithm.
dens_frac: float (optional, default 0.3)
Controls the fraction of epochs (between 0 and 1) where the
density-augmented objective is used in densMAP. The first
(1 - dens_frac) fraction of epochs optimize the original UMAP objective
before introducing the density correlation term.
dens_var_shift: float (optional, default 0.1)
A small constant added to the variance of local radii in the
embedding when calculating the density correlation objective to
prevent numerical instability from dividing by a small number
output_dens: float (optional, default False)
Determines whether the local radii of the final embedding (an inverse
measure of local density) are computed and returned in addition to
the embedding. If set to True, local radii of the original data
are also included in the output for comparison; the output is a tuple
(embedding, original local radii, embedding local radii). This option
can also be used when densmap=False to calculate the densities for
UMAP embeddings.
disconnection_distance: float (optional, default np.inf or maximal value for bounded distances)
Disconnect any vertices of distance greater than or equal to disconnection_distance when approximating the
manifold via our k-nn graph. This is particularly useful in the case that you have a bounded metric. The
UMAP assumption that we have a connected manifold can be problematic when you have points that are maximally
different from all the rest of your data. The connected manifold assumption will make such points have perfect
similarity to a random set of other points. Too many such points will artificially connect your space.
precomputed_knn: tuple (optional, default (None,None,None))
If the k-nearest neighbors of each point has already been calculated you
can pass them in here to save computation time. The number of nearest
neighbors in the precomputed_knn must be greater or equal to the
n_neighbors parameter. This should be a tuple containing the output
of the nearest_neighbors() function or attributes from a previously fit
UMAP object; (knn_indices, knn_dists, knn_search_index). If you wish to use
k-nearest neighbors data calculated by another package then provide a tuple of
the form (knn_indices, knn_dists). The contents of the tuple should be two numpy
arrays of shape (N, n_neighbors) where N is the number of items in the
input data. The first array should be the integer indices of the nearest
neighbors, and the second array should be the corresponding distances. The
nearest neighbor of each item should be itself, e.g. the nearest neighbor of
item 0 should be 0, the nearest neighbor of item 1 is 1 and so on. Please note
that you will *not* be able to transform new data in this case.
"""
def __init__(
self,
n_neighbors=15,
n_components=2,
metric="euclidean",
metric_kwds=None,
output_metric="euclidean",
output_metric_kwds=None,
n_epochs=None,
learning_rate=1.0,
init="spectral",
min_dist=0.1,
spread=1.0,
low_memory=True,
n_jobs=-1,
set_op_mix_ratio=1.0,
local_connectivity=1.0,
repulsion_strength=1.0,
negative_sample_rate=5,
transform_queue_size=4.0,
a=None,
b=None,
random_state=None,
angular_rp_forest=False,
target_n_neighbors=-1,
target_metric="categorical",
target_metric_kwds=None,
target_weight=0.5,
transform_seed=42,
transform_mode="embedding",
force_approximation_algorithm=False,
verbose=False,
tqdm_kwds=None,
unique=False,
densmap=False,
dens_lambda=2.0,
dens_frac=0.3,
dens_var_shift=0.1,
output_dens=False,
disconnection_distance=None,
precomputed_knn=(None, None, None),
):
self.n_neighbors = n_neighbors
self.metric = metric
self.output_metric = output_metric
self.target_metric = target_metric
self.metric_kwds = metric_kwds
self.output_metric_kwds = output_metric_kwds
self.n_epochs = n_epochs
self.init = init
self.n_components = n_components
self.repulsion_strength = repulsion_strength
self.learning_rate = learning_rate
self.spread = spread
self.min_dist = min_dist
self.low_memory = low_memory
self.set_op_mix_ratio = set_op_mix_ratio
self.local_connectivity = local_connectivity
self.negative_sample_rate = negative_sample_rate
self.random_state = random_state
self.angular_rp_forest = angular_rp_forest
self.transform_queue_size = transform_queue_size
self.target_n_neighbors = target_n_neighbors
self.target_metric = target_metric
self.target_metric_kwds = target_metric_kwds
self.target_weight = target_weight
self.transform_seed = transform_seed
self.transform_mode = transform_mode
self.force_approximation_algorithm = force_approximation_algorithm
self.verbose = verbose
self.tqdm_kwds = tqdm_kwds
self.unique = unique
self.densmap = densmap
self.dens_lambda = dens_lambda
self.dens_frac = dens_frac
self.dens_var_shift = dens_var_shift
self.output_dens = output_dens
self.disconnection_distance = disconnection_distance
self.precomputed_knn = precomputed_knn
self.n_jobs = n_jobs
self.a = a
self.b = b
def _validate_parameters(self):
if self.set_op_mix_ratio < 0.0 or self.set_op_mix_ratio > 1.0:
raise ValueError("set_op_mix_ratio must be between 0.0 and 1.0")
if self.repulsion_strength < 0.0:
raise ValueError("repulsion_strength cannot be negative")
if self.min_dist > self.spread:
raise ValueError("min_dist must be less than or equal to spread")
if self.min_dist < 0.0:
raise ValueError("min_dist cannot be negative")
if not isinstance(self.init, str) and not isinstance(self.init, np.ndarray):
raise ValueError("init must be a string or ndarray")
if isinstance(self.init, str) and self.init not in (
"pca",
"spectral",
"random",
"tswspectral",
):
raise ValueError(
'string init values must be one of: "pca", "tswspectral",'
' "spectral" or "random"'
)
if (
isinstance(self.init, np.ndarray)
and self.init.shape[1] != self.n_components
):
raise ValueError("init ndarray must match n_components value")
if not isinstance(self.metric, str) and not callable(self.metric):
raise ValueError("metric must be string or callable")
if self.negative_sample_rate < 0:
raise ValueError("negative sample rate must be positive")
if self._initial_alpha < 0.0:
raise ValueError("learning_rate must be positive")
if self.n_neighbors < 2:
raise ValueError("n_neighbors must be greater than 1")
if self.target_n_neighbors < 2 and self.target_n_neighbors != -1:
raise ValueError("target_n_neighbors must be greater than 1")
if not isinstance(self.n_components, int):
if isinstance(self.n_components, str):
raise ValueError("n_components must be an int")
if self.n_components % 1 != 0:
raise ValueError("n_components must be a whole number")
try:
# this will convert other types of int (eg. numpy int64)
# to Python int
self.n_components = int(self.n_components)
except ValueError:
raise ValueError("n_components must be an int")
if self.n_components < 1:
raise ValueError("n_components must be greater than 0")
self.n_epochs_list = None
if (
isinstance(self.n_epochs, list)
or isinstance(self.n_epochs, tuple)
or isinstance(self.n_epochs, np.ndarray)
):
if not issubclass(
np.array(self.n_epochs).dtype.type, np.integer
) or not np.all(np.array(self.n_epochs) >= 0):
raise ValueError(
"n_epochs must be a nonnegative integer "
"or a list of nonnegative integers"
)
self.n_epochs_list = list(self.n_epochs)
elif self.n_epochs is not None and (
self.n_epochs < 0 or not isinstance(self.n_epochs, int)
):
raise ValueError(
"n_epochs must be a nonnegative integer "
"or a list of nonnegative integers"
)
if self.metric_kwds is None:
self._metric_kwds = {}
else:
self._metric_kwds = self.metric_kwds
if self.output_metric_kwds is None:
self._output_metric_kwds = {}
else:
self._output_metric_kwds = self.output_metric_kwds
if self.target_metric_kwds is None:
self._target_metric_kwds = {}
else:
self._target_metric_kwds = self.target_metric_kwds
# check sparsity of data upfront to set proper _input_distance_func &
# save repeated checks later on
if scipy.sparse.isspmatrix_csr(self._raw_data):
self._sparse_data = True
else:
self._sparse_data = False
# set input distance metric & inverse_transform distance metric
if callable(self.metric):
in_returns_grad = self._check_custom_metric(
self.metric, self._metric_kwds, self._raw_data
)
if in_returns_grad:
_m = self.metric
@numba.njit(fastmath=True)
def _dist_only(x, y, *kwds):
return _m(x, y, *kwds)[0]
self._input_distance_func = _dist_only
self._inverse_distance_func = self.metric
else:
self._input_distance_func = self.metric
self._inverse_distance_func = None
warn(
"custom distance metric does not return gradient; inverse_transform will be unavailable. "
"To enable using inverse_transform method, define a distance function that returns a tuple "
"of (distance [float], gradient [np.array])"
)
elif self.metric == "precomputed":
if self.unique:
raise ValueError("unique is poorly defined on a precomputed metric")
warn("using precomputed metric; inverse_transform will be unavailable")
self._input_distance_func = self.metric
self._inverse_distance_func = None
elif self.metric == "hellinger" and self._raw_data.min() < 0:
raise ValueError("Metric 'hellinger' does not support negative values")
elif self.metric in dist.named_distances:
if self._sparse_data:
if self.metric in sparse.sparse_named_distances:
self._input_distance_func = sparse.sparse_named_distances[
self.metric
]
else:
raise ValueError(
"Metric {} is not supported for sparse data".format(self.metric)
)
else:
self._input_distance_func = dist.named_distances[self.metric]
try:
self._inverse_distance_func = dist.named_distances_with_gradients[
self.metric
]
except KeyError:
warn(
"gradient function is not yet implemented for {} distance metric; "
"inverse_transform will be unavailable".format(self.metric)
)
self._inverse_distance_func = None
elif self.metric in pynn_named_distances:
if self._sparse_data:
if self.metric in pynn_sparse_named_distances:
self._input_distance_func = pynn_sparse_named_distances[self.metric]
else:
raise ValueError(
"Metric {} is not supported for sparse data".format(self.metric)
)
else:
self._input_distance_func = pynn_named_distances[self.metric]
warn(
"gradient function is not yet implemented for {} distance metric; "
"inverse_transform will be unavailable".format(self.metric)
)
self._inverse_distance_func = None
else:
raise ValueError("metric is neither callable nor a recognised string")
# set output distance metric
if callable(self.output_metric):
out_returns_grad = self._check_custom_metric(
self.output_metric, self._output_metric_kwds
)
if out_returns_grad:
self._output_distance_func = self.output_metric
else:
raise ValueError(
"custom output_metric must return a tuple of (distance [float], gradient [np.array])"
)
elif self.output_metric == "precomputed":
raise ValueError("output_metric cannnot be 'precomputed'")
elif self.output_metric in dist.named_distances_with_gradients:
self._output_distance_func = dist.named_distances_with_gradients[
self.output_metric
]
elif self.output_metric in dist.named_distances:
raise ValueError(
"gradient function is not yet implemented for {}.".format(
self.output_metric
)
)
else:
raise ValueError(
"output_metric is neither callable nor a recognised string"
)
# set angularity for NN search based on metric
if self.metric in (
"cosine",
"correlation",
"dice",
"jaccard",
"ll_dirichlet",
"hellinger",
):
self.angular_rp_forest = True
if self.n_jobs < -1 or self.n_jobs == 0:
raise ValueError("n_jobs must be a postive integer, or -1 (for all cores)")
if self.n_jobs != 1 and self.random_state is not None:
self.n_jobs = 1
warn(
f"n_jobs value {self.n_jobs} overridden to 1 by setting random_state. Use no seed for parallelism."
)
if self.dens_lambda < 0.0:
raise ValueError("dens_lambda cannot be negative")
if self.dens_frac < 0.0 or self.dens_frac > 1.0:
raise ValueError("dens_frac must be between 0.0 and 1.0")
if self.dens_var_shift < 0.0:
raise ValueError("dens_var_shift cannot be negative")
self._densmap_kwds = {
"lambda": self.dens_lambda if self.densmap else 0.0,
"frac": self.dens_frac if self.densmap else 0.0,
"var_shift": self.dens_var_shift,
"n_neighbors": self.n_neighbors,
}
if self.densmap:
if self.output_metric not in ("euclidean", "l2"):
raise ValueError(
"Non-Euclidean output metric not supported for densMAP."
)
# This will be used to prune all edges of greater than a fixed value from our knn graph.
# We have preset defaults described in DISCONNECTION_DISTANCES for our bounded measures.
# Otherwise a user can pass in their own value.
if self.disconnection_distance is None:
self._disconnection_distance = DISCONNECTION_DISTANCES.get(
self.metric, np.inf
)
elif isinstance(self.disconnection_distance, int) or isinstance(
self.disconnection_distance, float
):
self._disconnection_distance = self.disconnection_distance
else:
raise ValueError("disconnection_distance must either be None or a numeric.")
if self.tqdm_kwds is None:
self.tqdm_kwds = {}
else:
if isinstance(self.tqdm_kwds, dict) is False:
raise ValueError(
"tqdm_kwds must be a dictionary. Please provide valid tqdm "
"parameters as key value pairs. Valid tqdm parameters can be "
"found here: https://github.com/tqdm/tqdm#parameters"
)
if "desc" not in self.tqdm_kwds:
self.tqdm_kwds["desc"] = "Epochs completed"
if "bar_format" not in self.tqdm_kwds:
bar_f = "{desc}: {percentage:3.0f}%| {bar} {n_fmt}/{total_fmt} [{elapsed}]"
self.tqdm_kwds["bar_format"] = bar_f
if hasattr(self, "knn_dists") and self.knn_dists is not None:
if self.unique:
raise ValueError(
"unique is not currently available for " "precomputed_knn."
)
if not isinstance(self.knn_indices, np.ndarray):
raise ValueError("precomputed_knn[0] must be ndarray object.")
if not isinstance(self.knn_dists, np.ndarray):
raise ValueError("precomputed_knn[1] must be ndarray object.")
if self.knn_dists.shape != self.knn_indices.shape:
raise ValueError(
"precomputed_knn[0] and precomputed_knn[1]"
" must be numpy arrays of the same size."
)
# #848: warn but proceed if no search index is present
if not isinstance(self.knn_search_index, NNDescent):
warn(
"precomputed_knn[2] (knn_search_index) "
"is not an NNDescent object: transforming new data with transform "
"will be unavailable."
)
if self.knn_dists.shape[1] < self.n_neighbors:
warn(
"precomputed_knn has a lower number of neighbors than "
"n_neighbors parameter. precomputed_knn will be ignored"
" and the k-nn will be computed normally."
)
self.knn_indices = None
self.knn_dists = None
self.knn_search_index = None
elif self.knn_dists.shape[0] != self._raw_data.shape[0]:
warn(
"precomputed_knn has a different number of samples than the"
" data you are fitting. precomputed_knn will be ignored and"
"the k-nn will be computed normally."
)
self.knn_indices = None
self.knn_dists = None
self.knn_search_index = None
elif (
self.knn_dists.shape[0] < 4096
and not self.force_approximation_algorithm
):
# force_approximation_algorithm is irrelevant for pre-computed knn
# always set it to True which keeps downstream code paths working
self.force_approximation_algorithm = True
elif self.knn_dists.shape[1] > self.n_neighbors:
# if k for precomputed_knn larger than n_neighbors we simply prune it
self.knn_indices = self.knn_indices[:, : self.n_neighbors]
self.knn_dists = self.knn_dists[:, : self.n_neighbors]
def _check_custom_metric(self, metric, kwds, data=None):
# quickly check to determine whether user-defined
# self.metric/self.output_metric returns both distance and gradient
if data is not None:
# if checking the high-dimensional distance metric, test directly on
# input data so we don't risk violating any assumptions potentially
# hard-coded in the metric (e.g., bounded; non-negative)
x, y = data[np.random.randint(0, data.shape[0], 2)]
else:
# if checking the manifold distance metric, simulate some data on a
# reasonable interval with output dimensionality
x, y = np.random.uniform(low=-10, high=10, size=(2, self.n_components))
if scipy.sparse.issparse(data):
metric_out = metric(x.indices, x.data, y.indices, y.data, **kwds)
else:
metric_out = metric(x, y, **kwds)
# True if metric returns iterable of length 2, False otherwise
return hasattr(metric_out, "__iter__") and len(metric_out) == 2
def _populate_combined_params(self, *models):
self.n_neighbors = flattened([m.n_neighbors for m in models])
self.metric = flattened([m.metric for m in models])
self.metric_kwds = flattened([m.metric_kwds for m in models])
self.output_metric = flattened([m.output_metric for m in models])
self.n_epochs = flattened(
[m.n_epochs if m.n_epochs is not None else -1 for m in models]
)
if all([x == -1 for x in self.n_epochs]):
self.n_epochs = None
self.init = flattened([m.init for m in models])
self.n_components = flattened([m.n_components for m in models])
self.repulsion_strength = flattened([m.repulsion_strength for m in models])
self.learning_rate = flattened([m.learning_rate for m in models])
self.spread = flattened([m.spread for m in models])
self.min_dist = flattened([m.min_dist for m in models])
self.low_memory = flattened([m.low_memory for m in models])
self.set_op_mix_ratio = flattened([m.set_op_mix_ratio for m in models])
self.local_connectivity = flattened([m.local_connectivity for m in models])
self.negative_sample_rate = flattened([m.negative_sample_rate for m in models])
self.random_state = flattened([m.random_state for m in models])
self.angular_rp_forest = flattened([m.angular_rp_forest for m in models])
self.transform_queue_size = flattened([m.transform_queue_size for m in models])
self.target_n_neighbors = flattened([m.target_n_neighbors for m in models])
self.target_metric = flattened([m.target_metric for m in models])
self.target_metric_kwds = flattened([m.target_metric_kwds for m in models])
self.target_weight = flattened([m.target_weight for m in models])
self.transform_seed = flattened([m.transform_seed for m in models])
self.force_approximation_algorithm = flattened(
[m.force_approximation_algorithm for m in models]
)
self.verbose = flattened([m.verbose for m in models])
self.unique = flattened([m.unique for m in models])
self.densmap = flattened([m.densmap for m in models])
self.dens_lambda = flattened([m.dens_lambda for m in models])
self.dens_frac = flattened([m.dens_frac for m in models])
self.dens_var_shift = flattened([m.dens_var_shift for m in models])
self.output_dens = flattened([m.output_dens for m in models])
self.a = flattened([m.a for m in models])
self.b = flattened([m.b for m in models])
self._a = flattened([m._a for m in models])
self._b = flattened([m._b for m in models])
def __mul__(self, other):
check_is_fitted(
self, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
check_is_fitted(
other, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
if self.graph_.shape[0] != other.graph_.shape[0]:
raise ValueError("Only models with the equivalent samples can be combined")
result = UMAP()
result._populate_combined_params(self, other)
result.graph_ = general_simplicial_set_intersection(
self.graph_, other.graph_, 0.5
)
result.graph_ = reset_local_connectivity(result.graph_, True)
if scipy.sparse.csgraph.connected_components(result.graph_)[0] > 1:
warn(
"Combined graph is not connected but multi-component layout is unsupported. "
"Falling back to random initialization."
)
init = "random"
else:
init = "spectral"
result.densmap = np.any(result.densmap)
result.output_dens = np.any(result.output_dens)
result._densmap_kwds = {
"lambda": np.max(result.dens_lambda),
"frac": np.max(result.dens_frac),
"var_shift": np.max(result.dens_var_shift),
"n_neighbors": np.max(result.n_neighbors),
}
if result.n_epochs is None:
n_epochs = None
else:
n_epochs = np.max(result.n_epochs)
result.embedding_, aux_data = simplicial_set_embedding(
None,
result.graph_,
np.min(result.n_components),
np.min(result.learning_rate),
np.mean(result._a),
np.mean(result._b),
np.mean(result.repulsion_strength),
np.mean(result.negative_sample_rate),
n_epochs,
init,
check_random_state(42),
"euclidean",
{},
result.densmap,
result._densmap_kwds,
result.output_dens,
parallel=False,
verbose=bool(np.max(result.verbose)),
tqdm_kwds=self.tqdm_kwds,
)
if result.output_dens:
result.rad_orig_ = aux_data["rad_orig"]
result.rad_emb_ = aux_data["rad_emb"]
return result
def __add__(self, other):
check_is_fitted(
self, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
check_is_fitted(
other, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
if self.graph_.shape[0] != other.graph_.shape[0]:
raise ValueError("Only models with the equivalent samples can be combined")
result = UMAP()
result._populate_combined_params(self, other)
result.graph_ = general_simplicial_set_union(self.graph_, other.graph_)
result.graph_ = reset_local_connectivity(result.graph_, True)
if scipy.sparse.csgraph.connected_components(result.graph_)[0] > 1:
warn(
"Combined graph is not connected but mult-component layout is unsupported. "
"Falling back to random initialization."
)
init = "random"
else:
init = "spectral"
result.densmap = np.any(result.densmap)
result.output_dens = np.any(result.output_dens)
result._densmap_kwds = {
"lambda": np.max(result.dens_lambda),
"frac": np.max(result.dens_frac),
"var_shift": np.max(result.dens_var_shift),
"n_neighbors": np.max(result.n_neighbors),
}
if result.n_epochs is None:
n_epochs = None
else:
n_epochs = np.max(result.n_epochs)
result.embedding_, aux_data = simplicial_set_embedding(
None,
result.graph_,
np.min(result.n_components),
np.min(result.learning_rate),
np.mean(result._a),
np.mean(result._b),
np.mean(result.repulsion_strength),
np.mean(result.negative_sample_rate),
n_epochs,
init,
check_random_state(42),
"euclidean",
{},
result.densmap,
result._densmap_kwds,
result.output_dens,
parallel=False,
verbose=bool(np.max(result.verbose)),
tqdm_kwds=self.tqdm_kwds,
)
if result.output_dens:
result.rad_orig_ = aux_data["rad_orig"]
result.rad_emb_ = aux_data["rad_emb"]
return result
def __sub__(self, other):
check_is_fitted(
self, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
check_is_fitted(
other, attributes=["graph_"], msg="Only fitted UMAP models can be combined"
)
if self.graph_.shape[0] != other.graph_.shape[0]:
raise ValueError("Only models with the equivalent samples can be combined")
result = UMAP()
result._populate_combined_params(self, other)
result.graph_ = general_simplicial_set_intersection(
self.graph_, other.graph_, weight=0.5, right_complement=True
)
result.graph_ = reset_local_connectivity(result.graph_, False)
if scipy.sparse.csgraph.connected_components(result.graph_)[0] > 1:
warn(
"Combined graph is not connected but mult-component layout is unsupported. "
"Falling back to random initialization."
)
init = "random"
else:
init = "spectral"
result.densmap = np.any(result.densmap)
result.output_dens = np.any(result.output_dens)
result._densmap_kwds = {
"lambda": np.max(result.dens_lambda),
"frac": np.max(result.dens_frac),
"var_shift": np.max(result.dens_var_shift),
"n_neighbors": np.max(result.n_neighbors),
}
if result.n_epochs is None:
n_epochs = None
else:
n_epochs = np.max(result.n_epochs)
result.embedding_, aux_data = simplicial_set_embedding(
None,
result.graph_,
np.min(result.n_components),
np.min(result.learning_rate),
np.mean(result._a),
np.mean(result._b),
np.mean(result.repulsion_strength),
np.mean(result.negative_sample_rate),
n_epochs,
init,
check_random_state(42),
"euclidean",
{},
result.densmap,
result._densmap_kwds,
result.output_dens,
parallel=False,
verbose=bool(np.max(result.verbose)),
tqdm_kwds=self.tqdm_kwds,
)
if result.output_dens:
result.rad_orig_ = aux_data["rad_orig"]
result.rad_emb_ = aux_data["rad_emb"]
return result
def fit(self, X, y=None, ensure_all_finite=True, **kwargs):
"""Fit X into an embedded space.
Optionally use y for supervised dimension reduction.
Parameters
----------
X : array, shape (n_samples, n_features) or (n_samples, n_samples)
If the metric is 'precomputed' X must be a square distance
matrix. Otherwise it contains a sample per row. If the method
is 'exact', X may be a sparse matrix of type 'csr', 'csc'
or 'coo'.
y : array, shape (n_samples)
A target array for supervised dimension reduction. How this is
handled is determined by parameters UMAP was instantiated with.
The relevant attributes are ``target_metric`` and
``target_metric_kwds``.
ensure_all_finite : Whether to raise an error on np.inf, np.nan, pd.NA in array.
The possibilities are: - True: Force all values of array to be finite.
- False: accepts np.inf, np.nan, pd.NA in array.
- 'allow-nan': accepts only np.nan and pd.NA values in array.
Values cannot be infinite.
**kwargs : optional
Any additional keyword arguments are passed to _fit_embed_data.
"""
if self.metric in ("bit_hamming", "bit_jaccard"):
X = check_array(
X, dtype=np.uint8, order="C", ensure_all_finite=ensure_all_finite
)
else:
X = check_array(
X,
dtype=np.float32,
accept_sparse="csr",
order="C",
ensure_all_finite=ensure_all_finite,
)
self._raw_data = X
# Handle all the optional arguments, setting default
if self.a is None or self.b is None:
self._a, self._b = find_ab_params(self.spread, self.min_dist)
else:
self._a = self.a
self._b = self.b
if isinstance(self.init, np.ndarray):
init = check_array(
self.init,
dtype=np.float32,
accept_sparse=False,
ensure_all_finite=ensure_all_finite,
)
else:
init = self.init
self._initial_alpha = self.learning_rate
self.knn_indices = self.precomputed_knn[0]
self.knn_dists = self.precomputed_knn[1]
# #848: allow precomputed knn to not have a search index
if len(self.precomputed_knn) == 2:
self.knn_search_index = None
else:
self.knn_search_index = self.precomputed_knn[2]
self._validate_parameters()
if self.verbose:
print(str(self))
self._original_n_threads = numba.get_num_threads()
if self.n_jobs > 0 and self.n_jobs is not None:
numba.set_num_threads(self.n_jobs)
# Check if we should unique the data
# We've already ensured that we aren't in the precomputed case
if self.unique:
# check if the matrix is dense
if self._sparse_data:
# Call a sparse unique function
index, inverse, counts = csr_unique(X)
else:
index, inverse, counts = np.unique(
X,
return_index=True,
return_inverse=True,
return_counts=True,
axis=0,
)[1:4]
if self.verbose:
print(
"Unique=True -> Number of data points reduced from ",
X.shape[0],
" to ",
X[index].shape[0],
)
most_common = np.argmax(counts)
print(
"Most common duplicate is",
index[most_common],
" with a count of ",
counts[most_common],
)
# We'll expose an inverse map when unique=True for users to map from our internal structures to their data
self._unique_inverse_ = inverse
# If we aren't asking for unique use the full index.
# This will save special cases later.
else:
index = np.arange(X.shape[0])
inverse = np.arange(X.shape[0])
# Compute the indexed copy of X exactly once and reuse it everywhere
# below. Previously X[index] was re-materialized at each call site (and
# several times just to read .shape[0]), allocating the full N x D matrix
# repeatedly. A single copy is kept: it also preserves UMAP's contract of
# never mutating the caller's data (and self._raw_data) when a distance
# metric writes into the array it is handed.
n_index_samples = index.shape[0]
X_indexed = X[index]
# Error check n_neighbors based on data size
if n_index_samples <= self.n_neighbors:
if n_index_samples == 1:
self.embedding_ = np.zeros(
(1, self.n_components)
) # needed to sklearn comparability
return self
warn(
"n_neighbors is larger than the dataset size; truncating to "
"X.shape[0] - 1"
)
self._n_neighbors = n_index_samples - 1
if self.densmap:
self._densmap_kwds["n_neighbors"] = self._n_neighbors
else:
self._n_neighbors = self.n_neighbors
# Note: unless it causes issues for setting 'index', could move this to
# initial sparsity check above
if self._sparse_data and not X.has_sorted_indices:
X.sort_indices()
random_state = check_random_state(self.random_state)
if self.verbose:
print(ts(), "Construct fuzzy simplicial set")
if self.metric == "precomputed" and self._sparse_data:
# For sparse precomputed distance matrices, we just argsort the rows to find
# nearest neighbors. To make this easier, we expect matrices that are
# symmetrical (so we can find neighbors by looking at rows in isolation,
# rather than also having to consider that sample's column too).
# print("Computing KNNs for sparse precomputed distances...")
if sparse_tril(X).getnnz() != sparse_triu(X).getnnz():
raise ValueError(
"Sparse precomputed distance matrices should be symmetrical!"
)
if not np.all(X.diagonal() == 0):
raise ValueError("Non-zero distances from samples to themselves!")
if self.knn_dists is None:
self._knn_indices = np.zeros((X.shape[0], self.n_neighbors), dtype=int)
self._knn_dists = np.zeros(self._knn_indices.shape, dtype=float)
# X is CSR (check_array(accept_sparse="csr")), so slice its backing
# arrays directly instead of materializing a temporary matrix per row.
X_indptr, X_indices, X_data = X.indptr, X.indices, X.data
for row_id in range(X.shape[0]):
# Find KNNs row-by-row
row_start, row_end = X_indptr[row_id], X_indptr[row_id + 1]
row_data = X_data[row_start:row_end]
row_indices = X_indices[row_start:row_end]
if len(row_data) < self._n_neighbors:
raise ValueError(
"Some rows contain fewer than n_neighbors distances!"
)
# argpartition selects the k smallest in O(d) vs O(d·log d) for argsort
row_nn_data_indices = np.argpartition(row_data, self._n_neighbors)[
: self._n_neighbors
]
row_nn_data_indices = row_nn_data_indices[
np.argsort(row_data[row_nn_data_indices])
]
self._knn_indices[row_id] = row_indices[row_nn_data_indices]
self._knn_dists[row_id] = row_data[row_nn_data_indices]
else:
self._knn_indices = self.knn_indices
self._knn_dists = self.knn_dists
# Disconnect any vertices farther apart than _disconnection_distance
disconnected_index = self._knn_dists >= self._disconnection_distance
self._knn_indices[disconnected_index] = -1
self._knn_dists[disconnected_index] = np.inf
edges_removed = disconnected_index.sum()
(
self.graph_,
self._sigmas,
self._rhos,
self.graph_dists_,
) = fuzzy_simplicial_set(
X_indexed,
self.n_neighbors,
random_state,
"precomputed",
self._metric_kwds,
self._knn_indices,
self._knn_dists,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
self.densmap or self.output_dens,
)
# Report the number of vertices with degree 0 in our our umap.graph_
# This ensures that they were properly disconnected.
vertices_disconnected = np.sum(
np.array(self.graph_.sum(axis=1)).flatten() == 0
)
raise_disconnected_warning(
edges_removed,
vertices_disconnected,
self._disconnection_distance,
self._raw_data.shape[0],
verbose=self.verbose,
)
# Handle small cases efficiently by computing all distances
elif n_index_samples < 4096 and not self.force_approximation_algorithm:
self._small_data = True
try:
# sklearn pairwise_distances fails for callable metric on sparse data
_m = self.metric if self._sparse_data else self._input_distance_func
dmat = dist.numba_aware_pairwise_distances(
X_indexed, metric=_m, **self._metric_kwds
)
except (ValueError, TypeError) as e:
# metric is numba.jit'd or not supported by sklearn,
# fallback to pairwise special
if self._sparse_data:
# Get a fresh metric since we are casting to dense
if not callable(self.metric):
_m = dist.named_distances[self.metric]
dmat = dist.pairwise_special_metric(
X_indexed.toarray(),
metric=_m,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
X_indexed,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
X_indexed,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
# set any values greater than disconnection_distance to be np.inf.
# This will have no effect when _disconnection_distance is not set since it defaults to np.inf.
# The default (inf) makes the comparison an all-False no-op, so skip
# the two N x N boolean temporaries entirely; otherwise compute the
# mask once and reuse it for both the count and the in-place fill.
if np.isfinite(self._disconnection_distance):
disconnected_mask = dmat >= self._disconnection_distance
edges_removed = disconnected_mask.sum()
dmat[disconnected_mask] = np.inf
else:
edges_removed = 0
(
self.graph_,
self._sigmas,
self._rhos,
self.graph_dists_,
) = fuzzy_simplicial_set(
dmat,
self._n_neighbors,
random_state,
"precomputed",
self._metric_kwds,
None,
None,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
self.densmap or self.output_dens,
)
# Report the number of vertices with degree 0 in our umap.graph_
# This ensures that they were properly disconnected.
vertices_disconnected = np.sum(
np.array(self.graph_.sum(axis=1)).flatten() == 0
)
raise_disconnected_warning(
edges_removed,
vertices_disconnected,
self._disconnection_distance,
self._raw_data.shape[0],
verbose=self.verbose,
)
else:
# Standard case
self._small_data = False
# Standard case
if self._sparse_data and self.metric in pynn_sparse_named_distances:
nn_metric = self.metric
elif not self._sparse_data and self.metric in pynn_named_distances:
nn_metric = self.metric
else:
nn_metric = self._input_distance_func
if self.knn_dists is None:
(
self._knn_indices,
self._knn_dists,
self._knn_search_index,
) = nearest_neighbors(
X_indexed,
self._n_neighbors,
nn_metric,
self._metric_kwds,
self.angular_rp_forest,
random_state,
self.low_memory,
use_pynndescent=True,
n_jobs=self.n_jobs,
verbose=self.verbose,
)
else:
self._knn_indices = self.knn_indices
self._knn_dists = self.knn_dists
self._knn_search_index = self.knn_search_index
# Disconnect any vertices farther apart than _disconnection_distance
disconnected_index = self._knn_dists >= self._disconnection_distance
self._knn_indices[disconnected_index] = -1
self._knn_dists[disconnected_index] = np.inf
edges_removed = disconnected_index.sum()
(
self.graph_,
self._sigmas,
self._rhos,
self.graph_dists_,
) = fuzzy_simplicial_set(
X_indexed,
self.n_neighbors,
random_state,
nn_metric,
self._metric_kwds,
self._knn_indices,
self._knn_dists,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
self.densmap or self.output_dens,
)
# Report the number of vertices with degree 0 in our umap.graph_
# This ensures that they were properly disconnected.
vertices_disconnected = np.sum(
np.array(self.graph_.sum(axis=1)).flatten() == 0
)
raise_disconnected_warning(
edges_removed,
vertices_disconnected,
self._disconnection_distance,
self._raw_data.shape[0],
verbose=self.verbose,
)
# Currently not checking if any duplicate points have differing labels
# Might be worth throwing a warning...
if y is not None:
len_X = len(X) if not self._sparse_data else X.shape[0]
if len_X != len(y):
raise ValueError(
"Length of x = {len_x}, length of y = {len_y}, while it must be equal.".format(
len_x=len_X, len_y=len(y)
)
)
if self.target_metric == "string":
y_ = y[index]
else:
y_ = check_array(
y, ensure_2d=False, ensure_all_finite=ensure_all_finite
)[index]
if self.target_metric == "categorical":
if self.target_weight < 1.0:
far_dist = 2.5 * (1.0 / (1.0 - self.target_weight))
else:
far_dist = 1.0e12
self.graph_ = discrete_metric_simplicial_set_intersection(
self.graph_, y_, far_dist=far_dist
)
elif self.target_metric in dist.DISCRETE_METRICS:
if self.target_weight < 1.0:
scale = 2.5 * (1.0 / (1.0 - self.target_weight))
else:
scale = 1.0e12
# self.graph_ = discrete_metric_simplicial_set_intersection(
# self.graph_,
# y_,
# metric=self.target_metric,
# metric_kws=self.target_metric_kwds,
# metric_scale=scale
# )
metric_kws = dist.get_discrete_params(y_, self.target_metric)
self.graph_ = discrete_metric_simplicial_set_intersection(
self.graph_,
y_,
metric=self.target_metric,
metric_kws=metric_kws,
metric_scale=scale,
)
else:
if len(y_.shape) == 1:
y_ = y_.reshape(-1, 1)
if self.target_n_neighbors == -1:
target_n_neighbors = self._n_neighbors
else:
target_n_neighbors = self.target_n_neighbors
# Handle the small case as precomputed as before
if y.shape[0] < 4096:
try:
ydmat = pairwise_distances(
y_, metric=self.target_metric, **self._target_metric_kwds
)
except (TypeError, ValueError):
ydmat = dist.pairwise_special_metric(
y_,
metric=self.target_metric,
kwds=self._target_metric_kwds,
ensure_all_finite=ensure_all_finite,
)
(
target_graph,
target_sigmas,
target_rhos,
) = fuzzy_simplicial_set(
ydmat,
target_n_neighbors,
random_state,
"precomputed",
self._target_metric_kwds,
None,
None,
False,
1.0,
1.0,
False,
)
else:
# Standard case
(
target_graph,
target_sigmas,
target_rhos,
) = fuzzy_simplicial_set(
y_,
target_n_neighbors,
random_state,
self.target_metric,
self._target_metric_kwds,
None,
None,
False,
1.0,
1.0,
False,
)
# product = self.graph_.multiply(target_graph)
# # self.graph_ = 0.99 * product + 0.01 * (self.graph_ +
# # target_graph -
# # product)
# self.graph_ = product
self.graph_ = general_simplicial_set_intersection(
self.graph_, target_graph, self.target_weight
)
self.graph_ = reset_local_connectivity(self.graph_)
self._supervised = True
else:
self._supervised = False
if self.densmap or self.output_dens:
self._densmap_kwds["graph_dists"] = self.graph_dists_
if self.verbose:
print(ts(), "Construct embedding")
if self.transform_mode == "embedding":
epochs = (
self.n_epochs_list if self.n_epochs_list is not None else self.n_epochs
)
# Use a fresh copy of the (unmutated) raw data here, independent of
# X_indexed: a distance metric that writes into the array it is
# handed could otherwise corrupt the data PCA-init / densMAP read.
self.embedding_, aux_data = self._fit_embed_data(
self._raw_data[index],
epochs,
init,
random_state, # JH why raw data?
**kwargs,
)
if self.n_epochs_list is not None:
if "embedding_list" not in aux_data:
raise KeyError(
"No list of embedding were found in 'aux_data'. "
"It is likely the layout optimization function "
"doesn't support the list of int for 'n_epochs'."
)
else:
self.embedding_list_ = [
e[inverse] for e in aux_data["embedding_list"]
]
# Assign any points that are fully disconnected from our manifold(s) to have embedding
# coordinates of np.nan. These will be filtered by our plotting functions automatically.
# They also prevent users from being deceived a distance query to one of these points.
# Might be worth moving this into simplicial_set_embedding or _fit_embed_data
disconnected_vertices = np.array(self.graph_.sum(axis=1)).flatten() == 0
if len(disconnected_vertices) > 0:
self.embedding_[disconnected_vertices] = np.full(
self.n_components, np.nan
)
self.embedding_ = self.embedding_[inverse]
if self.output_dens:
self.rad_orig_ = aux_data["rad_orig"][inverse]
self.rad_emb_ = aux_data["rad_emb"][inverse]
if self.verbose:
print(ts() + " Finished embedding")
numba.set_num_threads(self._original_n_threads)
self._input_hash = joblib.hash(self._raw_data)
if self.transform_mode == "embedding":
# Set number of features out for sklearn API
self._n_features_out = self.embedding_.shape[1]
else:
self._n_features_out = self.graph_.shape[1]
return self
def _fit_embed_data(self, X, n_epochs, init, random_state, **kwargs):
"""A method wrapper for simplicial_set_embedding that can be
replaced by subclasses. Arbitrary keyword arguments can be passed
through .fit() and .fit_transform().
"""
return simplicial_set_embedding(
X,
self.graph_,
self.n_components,
self._initial_alpha,
self._a,
self._b,
self.repulsion_strength,
self.negative_sample_rate,
n_epochs,
init,
random_state,
self._input_distance_func,
self._metric_kwds,
self.densmap,
self._densmap_kwds,
self.output_dens,
self._output_distance_func,
self._output_metric_kwds,
self.output_metric in ("euclidean", "l2"),
self.random_state is None,
self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
def fit_transform(self, X, y=None, ensure_all_finite=True, **kwargs):
"""Fit X into an embedded space and return that transformed
output.
Parameters
----------
X : array, shape (n_samples, n_features) or (n_samples, n_samples)
If the metric is 'precomputed' X must be a square distance
matrix. Otherwise it contains a sample per row.
y : array, shape (n_samples)
A target array for supervised dimension reduction. How this is
handled is determined by parameters UMAP was instantiated with.
The relevant attributes are ``target_metric`` and
``target_metric_kwds``.
ensure_all_finite : Whether to raise an error on np.inf, np.nan, pd.NA in array.
The possibilities are: - True: Force all values of array to be finite.
- False: accepts np.inf, np.nan, pd.NA in array.
- 'allow-nan': accepts only np.nan and pd.NA values in array.
Values cannot be infinite.
**kwargs : Any additional keyword arguments are passed to _fit_embed_data.
Returns
-------
X_new : array, shape (n_samples, n_components)
Embedding of the training data in low-dimensional space.
or a tuple (X_new, r_orig, r_emb) if ``output_dens`` flag is set,
which additionally includes:
r_orig: array, shape (n_samples)
Local radii of data points in the original data space (log-transformed).
r_emb: array, shape (n_samples)
Local radii of data points in the embedding (log-transformed).
"""
_input_dtype = getattr(X, "dtype", None)
self.fit(X, y, ensure_all_finite, **kwargs)
if self.transform_mode == "embedding":
embedding = self.embedding_
if _input_dtype is not None and np.issubdtype(_input_dtype, np.floating):
embedding = embedding.astype(_input_dtype, copy=False)
if self.output_dens:
return embedding, self.rad_orig_, self.rad_emb_
else:
return embedding
elif self.transform_mode == "graph":
return self.graph_
else:
raise ValueError(
"Unrecognized transform mode {}; should be one of 'embedding' or 'graph'".format(
self.transform_mode
)
)
def transform(self, X, ensure_all_finite=True):
"""Transform X into the existing embedded space and return that
transformed output.
Parameters
----------
X : array, shape (n_samples, n_features)
New data to be transformed.
ensure_all_finite : Whether to raise an error on np.inf, np.nan, pd.NA in array.
The possibilities are: - True: Force all values of array to be finite.
- False: accepts np.inf, np.nan, pd.NA in array.
- 'allow-nan': accepts only np.nan and pd.NA values in array.
Values cannot be infinite.
Returns
-------
X_new : array, shape (n_samples, n_components)
Embedding of the new data in low-dimensional space.
"""
_input_dtype = getattr(X, "dtype", None)
# If we fit just a single instance then error
if self._raw_data.shape[0] == 1:
raise ValueError(
"Transform unavailable when model was fit with only a single data sample."
)
# If we just have the original input then short circuit things
if self.metric in ("bit_hamming", "bit_jaccard"):
X = check_array(
X, dtype=np.uint8, order="C", ensure_all_finite=ensure_all_finite
)
else:
X = check_array(
X,
dtype=np.float32,
accept_sparse="csr",
order="C",
ensure_all_finite=ensure_all_finite,
)
# Match the sparsity of the data the model was fit on; the distance
# computations below assume both have the same format
if self.metric != "precomputed":
if self._sparse_data and not scipy.sparse.issparse(X):
X = scipy.sparse.csr_matrix(X)
elif not self._sparse_data and scipy.sparse.issparse(X):
X = X.toarray()
x_hash = joblib.hash(X)
if x_hash == self._input_hash:
if self.transform_mode == "embedding":
embedding = self.embedding_
if _input_dtype is not None and np.issubdtype(
_input_dtype, np.floating
):
embedding = embedding.astype(_input_dtype, copy=False)
return embedding
elif self.transform_mode == "graph":
return self.graph_
else:
raise ValueError(
"Unrecognized transform mode {}; should be one of 'embedding' or 'graph'".format(
self.transform_mode
)
)
if self.densmap:
raise NotImplementedError(
"Transforming data into an existing embedding not supported for densMAP."
)
# #848: knn_search_index is allowed to be None if not transforming new data,
# so now we must validate that if it exists it is not None.
# #1194: a precomputed metric never has a search index (nearest_neighbors
# returns None for it), and the precomputed branch below does not need
# one, so only enforce this for the metrics that query the index.
if (
self.metric != "precomputed"
and hasattr(self, "_knn_search_index")
and self._knn_search_index is None
):
raise NotImplementedError(
"No search index available: transforming data"
" into an existing embedding is not supported"
)
# X = check_array(X, dtype=np.float32, order="C", accept_sparse="csr")
random_state = check_random_state(self.transform_seed)
rng_state = random_state.randint(INT32_MIN, INT32_MAX, 3).astype(np.int64)
if self.metric == "precomputed":
warn(
"Transforming new data with precomputed metric. "
"We are assuming the input data is a matrix of distances from the new points "
"to the points in the training set. If the input matrix is sparse, it should "
"contain distances from the new points to their nearest neighbours "
"or approximate nearest neighbours in the training set."
)
assert X.shape[1] == self._raw_data.shape[0]
if scipy.sparse.issparse(X):
indices = np.full(
(X.shape[0], self._n_neighbors), dtype=np.int32, fill_value=-1
)
dists = np.full_like(indices, dtype=np.float32, fill_value=-1)
# X is CSR (check_array(accept_sparse="csr")), so slice its backing
# arrays directly instead of materializing a temporary matrix per row.
X_indptr, X_indices, X_data = X.indptr, X.indices, X.data
for i in range(X.shape[0]):
row_start, row_end = X_indptr[i], X_indptr[i + 1]
row_data = X_data[row_start:row_end]
row_indices = X_indices[row_start:row_end]
if len(row_data) < self._n_neighbors:
raise ValueError(
f"Need at least n_neighbors ({self.n_neighbors}) distances for each row!"
)
row_nn_data_indices = np.argpartition(row_data, self._n_neighbors)[
: self._n_neighbors
]
row_nn_data_indices = row_nn_data_indices[
np.argsort(row_data[row_nn_data_indices])
]
indices[i] = row_indices[row_nn_data_indices]
dists[i] = row_data[row_nn_data_indices]
else:
indices = np.argpartition(X, self._n_neighbors, axis=1)[
:, : self._n_neighbors
]
dists = np.take_along_axis(X, indices, axis=1)
sorted_idx = np.argsort(dists, axis=1)
indices = np.take_along_axis(indices, sorted_idx, axis=1).astype(
np.int32
)
dists = np.take_along_axis(dists, sorted_idx, axis=1)
assert np.min(indices) >= 0 and np.min(dists) >= 0.0
elif self._small_data:
try:
# sklearn pairwise_distances fails for callable metric on sparse data
_m = self.metric if self._sparse_data else self._input_distance_func
dmat = pairwise_distances(
X, self._raw_data, metric=_m, **self._metric_kwds
)
except (TypeError, ValueError):
# metric is numba.jit'd or not supported by sklearn,
# fallback to pairwise special
if self._sparse_data:
# Get a fresh metric since we are casting to dense
if not callable(self.metric):
_m = dist.named_distances[self.metric]
dmat = dist.pairwise_special_metric(
X.toarray(),
self._raw_data.toarray(),
metric=_m,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
X,
self._raw_data,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
X,
self._raw_data,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
# Select the k nearest in row-blocks so the argpartition index
# temporary is (block x N) rather than a full (M x N) int64 array
# kept alive by the [:, :k] view. Per-row results are identical to
# the full-matrix argpartition/argsort (rows are independent).
k = self._n_neighbors
n_t = dmat.shape[0]
indices = np.empty((n_t, k), dtype=np.int64)
dists = np.empty((n_t, k), dtype=dmat.dtype)
block = 1024
for s in range(0, n_t, block):
e = min(s + block, n_t)
d_block = dmat[s:e]
part = np.argpartition(d_block, k, axis=1)[:, :k]
r = np.arange(e - s)[:, None]
bd = d_block[r, part]
order = np.argsort(bd, axis=1)
indices[s:e] = part[r, order]
dists[s:e] = bd[r, order]
else:
epsilon = 0.24 if self._knn_search_index._angular_trees else 0.12
indices, dists = self._knn_search_index.query(
X, self.n_neighbors, epsilon=epsilon
)
dists = dists.astype(np.float32, order="C")
# Remove any nearest neighbours who's distances are greater than our disconnection_distance
indices[dists >= self._disconnection_distance] = -1
adjusted_local_connectivity = max(0.0, self.local_connectivity - 1.0)
sigmas, rhos = smooth_knn_dist(
dists,
float(self._n_neighbors),
local_connectivity=float(adjusted_local_connectivity),
)
rows, cols, vals, dists = compute_membership_strengths(
indices, dists, sigmas, rhos, bipartite=True
)
graph = scipy.sparse.coo_matrix(
(vals, (rows, cols)), shape=(X.shape[0], self._raw_data.shape[0])
)
if self.transform_mode == "graph":
return graph
# This was a very specially constructed graph with constant degree.
# That lets us do fancy unpacking by reshaping the csr matrix indices
# and data. Doing so relies on the constant degree assumption!
# csr_graph = normalize(graph.tocsr(), norm="l1")
# inds = csr_graph.indices.reshape(X.shape[0], self._n_neighbors)
# weights = csr_graph.data.reshape(X.shape[0], self._n_neighbors)
# embedding = init_transform(inds, weights, self.embedding_)
# This is less fast code than the above numba.jit'd code.
# It handles the fact that our nearest neighbour graph can now contain variable numbers of vertices.
csr_graph = graph.tocsr()
csr_graph.eliminate_zeros()
embedding = init_graph_transform(csr_graph, self.embedding_)
if self.n_epochs is None:
# For smaller datasets we can use more epochs
if graph.shape[0] <= 10000:
n_epochs = 100
else:
n_epochs = 30
else:
# n_epochs may be a list of epochs at which to save intermediate
# embeddings; only the largest value matters here.
n_epochs = int(np.max(self.n_epochs) // 3.0)
graph.data[graph.data < (graph.data.max() / float(n_epochs))] = 0.0
graph.eliminate_zeros()
epochs_per_sample = make_epochs_per_sample(graph.data, n_epochs)
head = graph.row
tail = graph.col
weight = graph.data
# optimize_layout = make_optimize_layout(
# self._output_distance_func,
# tuple(self.output_metric_kwds.values()),
# )
if self.output_metric == "euclidean":
embedding = optimize_layout_euclidean(
embedding,
self.embedding_.astype(np.float32, copy=True), # Fixes #179 & #217,
head,
tail,
n_epochs,
graph.shape[1],
epochs_per_sample,
self._a,
self._b,
rng_state,
self.repulsion_strength,
self._initial_alpha / 4.0,
self.negative_sample_rate,
self.random_state is None,
verbose=self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
else:
embedding = optimize_layout_generic(
embedding,
self.embedding_.astype(np.float32, copy=True), # Fixes #179 & #217
head,
tail,
n_epochs,
graph.shape[1],
epochs_per_sample,
self._a,
self._b,
rng_state,
self.repulsion_strength,
self._initial_alpha / 4.0,
self.negative_sample_rate,
self._output_distance_func,
tuple(self._output_metric_kwds.values()),
verbose=self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
if _input_dtype is not None and np.issubdtype(_input_dtype, np.floating):
embedding = embedding.astype(_input_dtype, copy=False)
return embedding
def inverse_transform(self, X):
"""Transform X in the existing embedded space back into the input
data space and return that transformed output.
Parameters
----------
X : array, shape (n_samples, n_components)
New points to be inverse transformed.
Returns
-------
X_new : array, shape (n_samples, n_features)
Generated data points new data in data space.
"""
if self._sparse_data:
raise ValueError("Inverse transform not available for sparse input.")
elif self._inverse_distance_func is None:
raise ValueError("Inverse transform not available for given metric.")
elif self.densmap:
raise ValueError("Inverse transform not available for densMAP.")
elif self.n_components >= 8:
warn(
"Inverse transform works best with low dimensional embeddings."
" Results may be poor, or this approach to inverse transform"
" may fail altogether! If you need a high dimensional latent"
" space and inverse transform operations consider using an"
" autoencoder."
)
elif self.transform_mode == "graph":
raise ValueError(
"Inverse transform not available for transform_mode = 'graph'"
)
X = check_array(X, dtype=np.float32, order="C")
random_state = check_random_state(self.transform_seed)
rng_state = random_state.randint(INT32_MIN, INT32_MAX, 3).astype(np.int64)
# build Delaunay complex (Does this not assume a roughly euclidean output metric)?
deltri = scipy.spatial.Delaunay(
self.embedding_, incremental=True, qhull_options="QJ"
)
neighbors = deltri.simplices[deltri.find_simplex(X)]
n_embed = self.embedding_.shape[0]
simplices = deltri.simplices
k = simplices.shape[1]
n_s = simplices.shape[0]
# All (vertex_a, vertex_b) pairs within each simplex — k² pairs × n_simplices
rows = np.empty(n_s * k * k, dtype=np.int32)
cols = np.empty(n_s * k * k, dtype=np.int32)
for a in range(k):
for b in range(k):
off = (a * k + b) * n_s
rows[off : off + n_s] = simplices[:, a]
cols[off : off + n_s] = simplices[:, b]
adjmat = scipy.sparse.csr_matrix(
(np.ones(n_s * k * k, dtype=np.int8), (rows, cols)),
shape=(n_embed, n_embed),
)
min_vertices = min(self._raw_data.shape[-1], self._raw_data.shape[0])
neighborhood = [
breadth_first_search(adjmat, v[0], min_vertices=min_vertices)
for v in neighbors
]
if callable(self.output_metric):
# need to create another numba.jit-able wrapper for callable
# output_metrics that return a tuple (already checked that it does
# during param validation in `fit` method)
_out_m = self.output_metric
@numba.njit(fastmath=True)
def _output_dist_only(x, y, *kwds):
return _out_m(x, y, *kwds)[0]
dist_only_func = _output_dist_only
elif self.output_metric in dist.named_distances.keys():
dist_only_func = dist.named_distances[self.output_metric]
else:
# shouldn't really ever get here because of checks already performed,
# but works as a failsafe in case attr was altered manually after fitting
raise ValueError(
"Unrecognized output metric: {}".format(self.output_metric)
)
dist_args = tuple(self._output_metric_kwds.values())
distances = [
np.array(
[
dist_only_func(X[i], self.embedding_[nb], *dist_args)
for nb in neighborhood[i]
]
)
for i in range(X.shape[0])
]
idx = np.array([np.argsort(e)[:min_vertices] for e in distances])
dists_output_space = np.array(
[distances[i][idx[i]] for i in range(len(distances))]
)
indices = np.array([neighborhood[i][idx[i]] for i in range(len(neighborhood))])
rows, cols, distances = np.array(
[
[i, indices[i, j], dists_output_space[i, j]]
for i in range(indices.shape[0])
for j in range(min_vertices)
]
).T
# calculate membership strength of each edge
weights = 1 / (1 + self._a * distances ** (2 * self._b))
# compute 1-skeleton
# convert 1-skeleton into coo_matrix adjacency matrix
graph = scipy.sparse.coo_matrix(
(weights, (rows, cols)), shape=(X.shape[0], self._raw_data.shape[0])
)
# That lets us do fancy unpacking by reshaping the csr matrix indices
# and data. Doing so relies on the constant degree assumption!
# csr_graph = graph.tocsr()
csr_graph = normalize(graph.tocsr(), norm="l1")
inds = csr_graph.indices.reshape(X.shape[0], min_vertices)
weights = csr_graph.data.reshape(X.shape[0], min_vertices)
inv_transformed_points = init_transform(inds, weights, self._raw_data)
if self.n_epochs is None:
# For smaller datasets we can use more epochs
if graph.shape[0] <= 10000:
n_epochs = 100
else:
n_epochs = 30
else:
# n_epochs may be a list of epochs at which to save intermediate
# embeddings; only the largest value matters here.
n_epochs = int(np.max(self.n_epochs) // 3.0)
# graph.data[graph.data < (graph.data.max() / float(n_epochs))] = 0.0
# graph.eliminate_zeros()
epochs_per_sample = make_epochs_per_sample(graph.data, n_epochs)
head = graph.row
tail = graph.col
weight = graph.data
inv_transformed_points = optimize_layout_inverse(
inv_transformed_points,
self._raw_data,
head,
tail,
weight,
self._sigmas,
self._rhos,
n_epochs,
graph.shape[1],
epochs_per_sample,
self._a,
self._b,
rng_state,
self.repulsion_strength,
self._initial_alpha / 4.0,
self.negative_sample_rate,
self._inverse_distance_func,
tuple(self._metric_kwds.values()),
verbose=self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
return inv_transformed_points
def update(self, X, ensure_all_finite=True):
if self.metric in ("bit_hamming", "bit_jaccard"):
X = check_array(
X, dtype=np.uint8, order="C", ensure_all_finite=ensure_all_finite
)
else:
X = check_array(
X,
dtype=np.float32,
accept_sparse="csr",
order="C",
ensure_all_finite=ensure_all_finite,
)
random_state = check_random_state(self.transform_seed)
rng_state = random_state.randint(INT32_MIN, INT32_MAX, 3).astype(np.int64)
original_size = self._raw_data.shape[0]
if self.metric == "precomputed":
raise ValueError("Update does not currently support precomputed metrics")
if self._supervised:
raise ValueError("Updating supervised models is not currently " "supported")
if self._small_data:
if self._sparse_data:
self._raw_data = scipy.sparse.vstack([self._raw_data, X])
else:
self._raw_data = np.vstack([self._raw_data, X])
if self._raw_data.shape[0] < 4096:
# still small data
try:
# sklearn pairwise_distances fails for callable metric on sparse data
_m = self.metric if self._sparse_data else self._input_distance_func
dmat = dist.numba_aware_pairwise_distances(
self._raw_data, metric=_m, **self._metric_kwds
)
except (ValueError, TypeError) as e:
# metric is numba.jit'd or not supported by sklearn,
# fallback to pairwise special
if self._sparse_data:
# Get a fresh metric since we are casting to dense
if not callable(self.metric):
_m = dist.named_distances[self.metric]
dmat = dist.pairwise_special_metric(
self._raw_data.toarray(),
metric=_m,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
self._raw_data,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
else:
dmat = dist.pairwise_special_metric(
self._raw_data,
metric=self._input_distance_func,
kwds=self._metric_kwds,
ensure_all_finite=ensure_all_finite,
)
self.graph_, self._sigmas, self._rhos = fuzzy_simplicial_set(
dmat,
self._n_neighbors,
random_state,
"precomputed",
self._metric_kwds,
None,
None,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
)
knn_indices = np.argsort(dmat)[:, : self.n_neighbors]
else:
# now large data
self._small_data = False
if self._sparse_data and self.metric in pynn_sparse_named_distances:
nn_metric = self.metric
elif not self._sparse_data and self.metric in pynn_named_distances:
nn_metric = self.metric
else:
nn_metric = self._input_distance_func
(
self._knn_indices,
self._knn_dists,
self._knn_search_index,
) = nearest_neighbors(
self._raw_data,
self._n_neighbors,
nn_metric,
self._metric_kwds,
self.angular_rp_forest,
random_state,
self.low_memory,
use_pynndescent=True,
n_jobs=self.n_jobs,
verbose=self.verbose,
)
self.graph_, self._sigmas, self._rhos = fuzzy_simplicial_set(
self._raw_data,
self.n_neighbors,
random_state,
nn_metric,
self._metric_kwds,
self._knn_indices,
self._knn_dists,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
)
knn_indices = self._knn_indices
init = np.zeros(
(self._raw_data.shape[0], self.n_components), dtype=np.float32
)
init[:original_size] = self.embedding_
init_update(init, original_size, knn_indices)
if self.n_epochs is None:
n_epochs = 0
else:
n_epochs = self.n_epochs
self.embedding_, aux_data = simplicial_set_embedding(
self._raw_data,
self.graph_,
self.n_components,
self._initial_alpha,
self._a,
self._b,
self.repulsion_strength,
self.negative_sample_rate,
n_epochs,
init,
random_state,
self._input_distance_func,
self._metric_kwds,
self.densmap,
self._densmap_kwds,
self.output_dens,
self._output_distance_func,
self._output_metric_kwds,
self.output_metric in ("euclidean", "l2"),
self.random_state is None,
self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
else:
self._knn_search_index.prepare()
self._knn_search_index.update(X)
self._raw_data = self._knn_search_index._raw_data
(
self._knn_indices,
self._knn_dists,
) = self._knn_search_index.neighbor_graph
if self._sparse_data and self.metric in pynn_sparse_named_distances:
nn_metric = self.metric
elif not self._sparse_data and self.metric in pynn_named_distances:
nn_metric = self.metric
else:
nn_metric = self._input_distance_func
self.graph_, self._sigmas, self._rhos = fuzzy_simplicial_set(
self._raw_data,
self.n_neighbors,
random_state,
nn_metric,
self._metric_kwds,
self._knn_indices,
self._knn_dists,
self.angular_rp_forest,
self.set_op_mix_ratio,
self.local_connectivity,
True,
self.verbose,
)
init = np.zeros(
(self._raw_data.shape[0], self.n_components), dtype=np.float32
)
init[:original_size] = self.embedding_
init_update(init, original_size, self._knn_indices)
if self.n_epochs is None:
n_epochs = 0
else:
n_epochs = self.n_epochs
self.embedding_, aux_data = simplicial_set_embedding(
self._raw_data,
self.graph_,
self.n_components,
self._initial_alpha,
self._a,
self._b,
self.repulsion_strength,
self.negative_sample_rate,
n_epochs,
init,
random_state,
self._input_distance_func,
self._metric_kwds,
self.densmap,
self._densmap_kwds,
self.output_dens,
self._output_distance_func,
self._output_metric_kwds,
self.output_metric in ("euclidean", "l2"),
self.random_state is None,
self.verbose,
tqdm_kwds=self.tqdm_kwds,
)
if self.output_dens:
self.rad_orig_ = aux_data["rad_orig"]
self.rad_emb_ = aux_data["rad_emb"]
def __repr__(self):
from sklearn.utils._pprint import _EstimatorPrettyPrinter
import re
pp = _EstimatorPrettyPrinter(
compact=True,
indent=1,
indent_at_name=True,
n_max_elements_to_show=50,
)
pp._changed_only = True
repr_ = pp.pformat(self)
repr_ = re.sub("tqdm_kwds={.*},", "", repr_, flags=re.S)
# remove empty lines
repr_ = re.sub("\n *\n", "\n", repr_, flags=re.S)
# remove extra whitespaces after a comma
repr_ = re.sub(", +", ", ", repr_)
return repr_

umap_.py at commit a2e06aa, under BSD-3-Clause · at the source

Overview

Authors: Wenjing Wang^1,2,3,4, Yaokai Yang^2,3,4, Qiong Liu^1,2,3,4, Yunming Gao^2,3,4, Qiantao Lv^2,3,4,5, Kaiqi Zhang^2,3,4, Jing Ning^2,3,4, Yi Sun^2,3,4

ORCID iDs: Wenjing Wang

Fudan University, Shanghai, China
Key Laboratory of Growth Regulation and Translational Research of Zhejiang Province, Research Center for Industries of the Future, School of Life Sciences, Westlake University, Hangzhou, China
Westlake Laboratory of Life Sciences and Biomedicine, Hangzhou, China
Institute of Basic Medical Sciences, Westlake Institute for Advanced Study, Hangzhou, China
Zhejiang University, Hangzhou, China

Institutions: Fudan University (China); Westlake University (China); Zhejiang University (China)

Journal: Frontiers in systems neuroscience, volume 20, article 1822122

Dates: received 3 March 2026; accepted 13 April 2026; published online 7 May 2026

Type: Research article · Language: English

License: CC BY

Identifiers: DOI 10.3389/fnsys.2026.1822122 · PMID 42181586 · PMCID PMC13190573 · OpenAlex W7160525263

Open access: gold, a free copy (OpenAlex)

Status: code verified (how the code is found and verified)

Categories: systems (subfield)

Methods: Connectivity, Statistics, Smoothing, state filtering, decompositions, Machine learning, Preprocessing

Keywords: connectomics, convergence-divergence circuit, dendritic computation, multimodal integration, superior protocerebrum, valence

Topic: Neurobiology and Insect Physiology Research (Cellular and Molecular Neuroscience, Neuroscience), according to OpenAlex

Funding: National Natural Science Foundation of China

Citations: not cited yet (Europe PMC); 195 references in the paper

Abstract

Valence detection in complex environment is critical for natural behaviors like foraging. Previous studies have explored valence processing in brain regions like lateral horn (LH) and mushroom body (MB) using simple synthetic stimuli in Drosophila. However, the neural basis for valence detection of natural objects in complex contexts remains unclear. Here, by brain-wide connectome analysis, we identified the evolutionarily conserved superior protocerebrum (SP) that integrates brain-wide multimodal inputs mainly via LH and MB, and sends widespread outputs particularly to the central complex (CX). This forms a convergence-divergence circuit resembling an autoencoder architecture, with SP as the bottleneck integrating multimodal information into low-dimensional valence signals. Specifically, SP input LH neurons integrate ethologically related innate valences for robust valence detection in natural environments, and the integration can be unimodal, such as that of diverse odors signaling food, or multimodal, such as that of wind and temperature signaling lousy weather. Opponent valences of attraction and aversion are further integrated into SP for complex valence detection. MB learned valences are also integrated into SP to update LH innate valences with recent experience for flexible valence detection. Attractive and aversive valences, either innate or learned, are integrated via excitatory and inhibitory synapses, respectively to form complex valence signals in a single SP neuron. Organized synaptic compartments support dendritic computation, with SP neurons exhibiting opposite synaptic organizations for opponent valences, indicating dendritic integration for complex valence detection. Our study highlights the importance of SP in multimodal opponent valence integration and suggests generalizable network and dendritic structures for complex valence processing.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repositories

Its files are read in the Code ↔ Paper reader above, with 8 matches between paragraphs and lines of code.

lmcinnes/umap

License: BSD-3-Clause

State: the link answers, verified on 28 October 2026

Evidence: files inventoried

Commit: a2e06aa18a5356e6a46a6a5fa619923b6ce12056, 26 October 2026

Languages: Python (49), Jupyter (11), Shell (3)

Size: 320 files, 63 scripts

Software Heritage: archived

Found in: the end of the paper

Holds: README, license file, environment (pyproject.toml, setup.py), tests, continuous integration, documentation, 11 notebooks

Not found: CITATION.cff

Tools: UMAP (51 files), NumPy (45 files), scikit-learn (39 files), Matplotlib (20 files), Numba (20 files), SciPy (18 files), TensorFlow (10 files), Keras (9 files), pandas (6 files), seaborn (6 files), h5py (1 file), Pillow (1 file), PyTorch (1 file), scikit-image (1 file)

Availability: 2 checks, the latest on 28 October 2026: the link answers

28 October 2026: the link answers
28 September 2026: the link answers

65 files

benchmarks/bench_adjmat_vectorize.py, Python, 98 lines
benchmarks/bench_fastmath_removal.py, Python, 301 lines
ci_scripts/install.sh, Shell, 86 lines
ci_scripts/success.sh, Shell, 13 lines
ci_scripts/test.sh, Shell, 12 lines
doc/bokeh_digits_plot.py, Python, 73 lines
doc/conf.py, Python, 239 lines
doc/plotting_example_interactive.py, Python, 32 lines
examples/digits/digits.py, Python, 34 lines
examples/galaxy10sdss.py, Python, 275 lines
examples/inverse_transform_example.py, Python, 60 lines
examples/iris/iris.py, Python, 37 lines
examples/mnist_torus_sphere_example.py, Python, 120 lines
examples/mnist_transform_new_data.py, Python, 48 lines
examples/plot_algorithm_comparison.py, Python, 140 lines
examples/plot_fashion-mnist_example.py, Python, 74 lines
examples/plot_feature_extraction_classification.py, Python, 72 lines
examples/plot_mnist_example.py, Python, 35 lines
notebooks/AnimatingUMAP.ipynb, Jupyter, 168 lines
notebooks/Document embedding using UMAP.ipynb, Jupyter, 163 lines
notebooks/MNIST_Landmarks.ipynb, Jupyter, 177 lines
notebooks/Parametric_UMAP/01.0-parametric-umap-mnist-embedding-basic.ipynb, Jupyter, 92 lines
notebooks/Parametric_UMAP/02.0-parametric-umap-mnist-embedding-convnet.ipynb, Jupyter, 89 lines
notebooks/Parametric_UMAP/03.0-parametric-umap-mnist-embedding-convnet-with-reconstruction.ipynb, Jupyter, 164 lines
notebooks/Parametric_UMAP/04.0-parametric-umap-mnist-embedding-convnet-with-autoencoder-loss.ipynb, Jupyter, 144 lines
notebooks/Parametric_UMAP/05.0-parametric-umap-with-callback.ipynb, Jupyter, 77 lines
notebooks/Parametric_UMAP/06.0-nonparametric-umap.ipynb, Jupyter, 60 lines
notebooks/Parametric_UMAP/07.0-parametric-umap-global-loss.ipynb, Jupyter, 225 lines
notebooks/UMAP usage and parameters.ipynb, Jupyter, 260 lines
setup.py, Python, 5 lines
umap/__init__.py, Python, 42 lines
umap/aligned_umap.py, Python, 611 lines
umap/distances.py, Python, 1,583 lines
umap/layouts.py, Python, 1,100 lines, 1 match
umap/parametric_umap.py, Python, 1,402 lines
umap/plot.py, Python, 1,721 lines
umap/sparse.py, Python, 624 lines
umap/spectral.py, Python, 601 lines
umap/tests/__init__.py, Python, 50 lines
umap/tests/conftest.py, Python, 232 lines
umap/tests/test_aligned_umap.py, Python, 188 lines
umap/tests/test_chunked_parallel_spatial_metric.py, Python, 333 lines
umap/tests/test_composite_models.py, Python, 117 lines
umap/tests/test_data_input.py, Python, 107 lines
umap/tests/test_densmap.py, Python, 80 lines
umap/tests/test_numba_aware_pairwise_distances.py, Python, 109 lines
umap/tests/test_pairwise_special_metric.py, Python, 110 lines
umap/tests/test_parametric_umap.py, Python, 190 lines
umap/tests/test_plot.py, Python, 74 lines
umap/tests/test_spectral.py, Python, 63 lines
umap/tests/test_umap.py, Python, 1 line
umap/tests/test_umap_get_feature_names_out.py, Python, 82 lines
umap/tests/test_umap_grads.py, Python, 324 lines
umap/tests/test_umap_metrics.py, Python, 759 lines
umap/tests/test_umap_nn.py, Python, 198 lines
umap/tests/test_umap_on_iris.py, Python, 335 lines
umap/tests/test_umap_ops.py, Python, 428 lines
umap/tests/test_umap_repeated_data.py, Python, 110 lines
umap/tests/test_umap_trustworthiness.py, Python, 172 lines
umap/tests/test_umap_validation_params.py, Python, 322 lines
umap/umap_.py, Python, 3,784 lines, 2 matches
umap/utils.py, Python, 225 lines
umap/validation.py, Python, 87 lines
LICENSE.txt, License, 29 lines
README.rst, Text, 571 lines

At the source: github.com/lmcinnes/umap

navis-org/navis

License: GPL-3.0

State: the link answers, verified on 28 October 2026

Evidence: files inventoried

Commit: 4fe56c87fb8c2b0c9f89ceb3a4beb6dbf8f30839, 24 October 2026

Languages: Python (295), JavaScript (2), Shell (2), Jupyter (1), Java (1)

Size: 470 files, 301 scripts

Software Heritage: archived

Found in: the end of the paper

Holds: README, license file, environment (pyproject.toml, requirements.txt, setup.py, binder/postBuild, binder/requirements.txt), tests, continuous integration, documentation, 1 notebook

Not found: CITATION.cff

Tools: NumPy (182 files), pandas (129 files), Matplotlib (57 files), SciPy (48 files), NetworkX (18 files), seaborn (12 files), Plotly (5 files), NEURON (4 files), scikit-image (4 files), h5py (3 files), igraph (2 files), scikit-learn (2 files), NiBabel (1 file), Numba (1 file), tifffile (1 file)

Availability: 2 checks, the latest on 28 October 2026: the link answers

28 October 2026: the link answers
26 September 2026: the link answers

303 files

conftest.py, Python, 33 lines
docs/examples/0_io/tutorial_io_00_skeletons.py, Python, 226 lines
docs/examples/0_io/tutorial_io_01_meshes.py, Python, 120 lines
docs/examples/0_io/tutorial_io_02_dotprops.py, Python, 143 lines
docs/examples/0_io/tutorial_io_03_r.py, Python, 192 lines
docs/examples/0_io/tutorial_io_04_pickle.py, Python, 54 lines
docs/examples/0_io/zzz_tutorial_io_05_skeletonize.py, Python, 252 lines
docs/examples/1a_plotting_general/tutorial_plotting_00_intro.py, Python, 354 lines
docs/examples/1a_plotting_general/tutorial_plotting_01_colors.py, Python, 307 lines
docs/examples/1b_plotting_2d/tutorial_plotting_2d_00_skeletons.py, Python, 195 lines
docs/examples/1b_plotting_2d/tutorial_plotting_2d_01_meshes.py, Python, 206 lines
docs/examples/1b_plotting_2d/tutorial_plotting_2d_02_volumes.py, Python, 178 lines
docs/examples/1c_plotting_3d/tutorial_plotting_3d_00_skeletons.py, Python, 201 lines
docs/examples/1c_plotting_3d/tutorial_plotting_3d_01_meshes.py, Python, 150 lines
docs/examples/1c_plotting_3d/tutorial_plotting_3d_02_volumes.py, Python, 161 lines
docs/examples/1d_plotting_misc/tutorial_plotting_misc_00_connectors.py, Python, 353 lines
docs/examples/1d_plotting_misc/tutorial_plotting_misc_01_barcode.py, Python, 100 lines
docs/examples/1d_plotting_misc/tutorial_plotting_misc_02_topology.py, Python, 129 lines
docs/examples/1d_plotting_misc/tutorial_plotting_misc_03_xkcd.py, Python, 59 lines
docs/examples/1d_plotting_misc/zzz_tutorial_plotting_misc_04_collage.py, Python, 378 lines
docs/examples/1e_plotting_examples/tutorial_plotting_ex_00_cortex.py, Python, 300 lines
docs/examples/2_morpho/tutorial_morpho_00_manipulate.py, Python, 451 lines
docs/examples/2_morpho/tutorial_morpho_01_analyze.py, Python, 338 lines
docs/examples/2_morpho/tutorial_morpho_02_label_prop.py, Python, 167 lines
docs/examples/2_morpho/tutorial_morpho_03_ad_split.py, Python, 204 lines
docs/examples/2_morpho/zzz_tutorial_morpho_05_ivscc.py, Python, 476 lines
docs/examples/2_morpho/zzz_tutorial_morpho_06_ivscc_em.py, Python, 567 lines
docs/examples/3_interfaces/tutorial_interfaces_00_neuron.py, Python, 332 lines
docs/examples/3_interfaces/tutorial_interfaces_01_neuron2.py, Python, 190 lines
docs/examples/3_interfaces/tutorial_interfaces_02_blender.py, Python, 165 lines
docs/examples/4_remote/tutorial_remote_00_neuprint.py, Python, 101 lines
docs/examples/4_remote/tutorial_remote_01_cloudvolume.py, Python, 110 lines
docs/examples/4_remote/tutorial_remote_02_microns.py, Python, 199 lines
docs/examples/4_remote/tutorial_remote_03_insect_db.py, Python, 122 lines
docs/examples/4_remote/tutorial_remote_04_h01.py, Python, 163 lines
docs/examples/4_remote/tutorial_remote_05_bil.py, Python, 99 lines
docs/examples/5_nblast/tutorial_nblast_00_intro.py, Python, 456 lines
docs/examples/5_nblast/tutorial_nblast_03_smat.py, Python, 147 lines
docs/examples/5_nblast/zzz_tutorial_nblast_01_flycircuit.py, Python, 159 lines
docs/examples/5_nblast/zzz_tutorial_nblast_02_hemibrain.py, Python, 160 lines
docs/examples/6_misc/tutorial_misc_00_multiprocess.py, Python, 444 lines
docs/examples/6_misc/tutorial_misc_01_transforms.py, Python, 471 lines
docs/examples/7_ml/tutorial_ml_00_normalize.py, Python, 209 lines
docs/examples/7_ml/tutorial_ml_01_sampling.py, Python, 486 lines
docs/examples/7_ml/tutorial_ml_02_augment.py, Python, 184 lines
docs/examples/tutorial_basic_00_basics.py, Python, 139 lines
docs/examples/tutorial_basic_01_neurons.py, Python, 625 lines
docs/examples/tutorial_basic_02_neuronlists.py, Python, 364 lines
docs/examples/tutorial_basic_03_masking.py, Python, 537 lines
docs/examples/tutorial_basic_04_attach.py, Python, 711 lines
docs/gallery_conf.py, Python, 26 lines
docs/javascripts/katex.js, JavaScript, 10 lines
docs/javascripts/mathjax.js, JavaScript, 19 lines
docs/tools/__init__.py, Python, 1 line
docs/tools/nb_to_doc.py, Python, 32 lines
download_test_data.sh, Shell, 21 lines
examples/colab.ipynb, Jupyter, 207 lines
main.py, Python, 24 lines
navis/__init__.py, Python, 41 lines
navis/__version__.py, Python, 15 lines
navis/_deprecated.py, Python, 321 lines
navis/compute/__init__.py, Python, 75 lines
navis/compute/backends/__init__.py, Python, 64 lines
navis/compute/backends/_dask.py, Python, 176 lines
navis/compute/backends/_joblib.py, Python, 147 lines
navis/compute/backends/_pathos.py, Python, 58 lines
navis/compute/backends/_submitit.py, Python, 163 lines
navis/compute/backends/base.py, Python, 769 lines
navis/compute/backends/local.py, Python, 184 lines
navis/compute/dispatch.py, Python, 641 lines
navis/compute/threads.py, Python, 188 lines
navis/config.py, Python, 337 lines
navis/conftest.py, Python, 27 lines
navis/connectivity/__init__.py, Python, 22 lines
navis/connectivity/adjacency.py, Python, 236 lines
navis/connectivity/cnmetrics.py, Python, 118 lines
navis/connectivity/matrix_utils.py, Python, 135 lines
navis/connectivity/predict.py, Python, 181 lines
navis/connectivity/similarity.py, Python, 494 lines
navis/conversion/__init__.py, Python, 19 lines
navis/conversion/converters.py, Python, 884 lines
navis/conversion/meshing.py, Python, 285 lines
navis/conversion/wrappers.py, Python, 204 lines
navis/core/__init__.py, Python, 35 lines
navis/core/base.py, Python, 1,781 lines
navis/core/core_utils.py, Python, 838 lines
navis/core/dotprop.py, Python, 713 lines
navis/core/masking.py, Python, 367 lines
navis/core/mesh.py, Python, 725 lines
navis/core/neuronlist.py, Python, 1,281 lines
navis/core/neurons.py, Python, 22 lines
navis/core/pipeline.py, Python, 824 lines
navis/core/schema.py, Python, 2,525 lines
navis/core/skeleton.py, Python, 1,626 lines
navis/core/volumes.py, Python, 738 lines
navis/core/voxel.py, Python, 1,503 lines
navis/data/__init__.py, Python, 17 lines
navis/data/load_data.py, Python, 220 lines
navis/graph/__init__.py, Python, 86 lines
navis/graph/clinic.py, Python, 246 lines
navis/graph/converters.py, Python, 926 lines, 1 match
navis/graph/graph_utils.py, Python, 3,635 lines
navis/interfaces/__init__.py, Python, 46 lines
navis/interfaces/allen_celltypes.py, Python, 119 lines
navis/interfaces/base.py, Python, 426 lines
navis/interfaces/blender.py, Python, 1,255 lines
navis/interfaces/brain_image_library.py, Python, 1,258 lines
navis/interfaces/cajal.py, Python, 910 lines
navis/interfaces/cave_utils.py, Python, 494 lines
navis/interfaces/h01.py, Python, 116 lines
navis/interfaces/insectbrain_db.py, Python, 736 lines
navis/interfaces/microns.py, Python, 205 lines
navis/interfaces/neuprint.py, Python, 1,003 lines
navis/interfaces/neuromorpho.py, Python, 317 lines
navis/interfaces/neuron/__init__.py, Python, 2 lines
navis/interfaces/neuron/comp.py, Python, 924 lines
navis/interfaces/neuron/network.py, Python, 861 lines
navis/interfaces/neuron/utils.py, Python, 39 lines
navis/interfaces/vfb.py, Python, 351 lines
navis/intersection/__init__.py, Python, 17 lines
navis/intersection/convex.py, Python, 55 lines
navis/intersection/intersect.py, Python, 408 lines
navis/intersection/ray.py, Python, 119 lines
navis/io/__init__.py, Python, 34 lines
navis/io/base.py, Python, 2,028 lines
navis/io/hdf_io.py, Python, 1,127 lines
navis/io/json_io.py, Python, 161 lines
navis/io/mesh_io.py, Python, 341 lines
navis/io/nmx_io.py, Python, 377 lines
navis/io/nrrd_io.py, Python, 376 lines
navis/io/pq_io.py, Python, 1,251 lines
navis/io/precomputed_io.py, Python, 686 lines
navis/io/rda_io.py, Python, 1,129 lines
navis/io/swc_io.py, Python, 755 lines
navis/io/tiff_io.py, Python, 294 lines
navis/matching/__init__.py, Python, 17 lines
navis/matching/bipartite.py, Python, 242 lines
navis/matching/pipeline.py, Python, 210 lines
navis/meshes/__init__.py, Python, 19 lines
navis/meshes/internals.py, Python, 260 lines
navis/meshes/mesh_utils.py, Python, 721 lines
navis/meshes/operations.py, Python, 339 lines
navis/ml/__init__.py, Python, 50 lines
navis/ml/augment.py, Python, 686 lines
navis/ml/chunk.py, Python, 1,098 lines
navis/ml/normalize.py, Python, 316 lines
navis/models/__init__.py, Python, 14 lines
navis/models/network_models.py, Python, 658 lines
navis/morpho/__init__.py, Python, 47 lines
navis/morpho/analyze.py, Python, 555 lines
navis/morpho/angles.py, Python, 370 lines
navis/morpho/caps.py, Python, 154 lines
navis/morpho/fq.py, Python, 163 lines
navis/morpho/heal.py, Python, 239 lines
navis/morpho/images.py, Python, 256 lines
navis/morpho/ivscc.py, Python, 840 lines
navis/morpho/manipulation.py, Python, 2,596 lines
navis/morpho/mmetrics.py, Python, 1,585 lines
navis/morpho/persistence.py, Python, 428 lines
navis/morpho/subset.py, Python, 582 lines
navis/nbl/__init__.py, Python, 35 lines
navis/nbl/ablast_funcs.py, Python, 551 lines
navis/nbl/backends/__init__.py, Python, 43 lines
navis/nbl/backends/base.py, Python, 216 lines
navis/nbl/backends/builtin.py, Python, 534 lines
navis/nbl/backends/fastcore.py, Python, 290 lines
navis/nbl/base.py, Python, 175 lines
navis/nbl/nblast_funcs.py, Python, 1,412 lines, 2 matches
navis/nbl/smat.py, Python, 1,141 lines
navis/nbl/synblast_funcs.py, Python, 395 lines, 2 matches
navis/nbl/utils.py, Python, 555 lines
navis/plotting/__init__.py, Python, 24 lines
navis/plotting/_common.py, Python, 436 lines
navis/plotting/collage.py, Python, 1,015 lines
navis/plotting/colors.py, Python, 805 lines
navis/plotting/d.py, Python, 194 lines
navis/plotting/dd.py, Python, 2,888 lines
navis/plotting/ddd.py, Python, 837 lines
navis/plotting/flat.py, Python, 541 lines
navis/plotting/k3d/__init__.py, Python, 1 line
navis/plotting/k3d/conftest.py, Python, 9 lines
navis/plotting/k3d/k3d_objects.py, Python, 455 lines
navis/plotting/plot_utils.py, Python, 796 lines
navis/plotting/plotly/__init__.py, Python, 1 line
navis/plotting/plotly/conftest.py, Python, 9 lines
navis/plotting/plotly/graph_objs.py, Python, 830 lines
navis/plotting/render.py, Python, 307 lines
navis/plotting/settings.py, Python, 297 lines
navis/plotting/viewer_utils.py, Python, 109 lines
navis/sampling/__init__.py, Python, 23 lines
navis/sampling/downsampling.py, Python, 486 lines
navis/sampling/points.py, Python, 803 lines
navis/sampling/resampling.py, Python, 990 lines
navis/sampling/utils.py, Python, 273 lines
navis/transforms/__init__.py, Python, 30 lines
navis/transforms/affine.py, Python, 119 lines
navis/transforms/align.py, Python, 786 lines
navis/transforms/backends.py, Python, 263 lines
navis/transforms/base.py, Python, 417 lines
navis/transforms/cmtk.py, Python, 945 lines
navis/transforms/elastix.py, Python, 466 lines
navis/transforms/factory.py, Python, 101 lines
navis/transforms/grid.py, Python, 324 lines
navis/transforms/h5reg.py, Python, 740 lines
navis/transforms/h5reg_java.py, Python, 215 lines
navis/transforms/h5reg_numba.py, Python, 298 lines
navis/transforms/images.py, Python, 207 lines
navis/transforms/jars/TransformCoordinates.java, Java, 84 lines
navis/transforms/moving_least_squares.py, Python, 236 lines
navis/transforms/similarity.py, Python, 246 lines
navis/transforms/templates.py, Python, 1,790 lines
navis/transforms/thinplate.py, Python, 291 lines
navis/transforms/xfm_funcs.py, Python, 540 lines
navis/utils/__init__.py, Python, 36 lines
navis/utils/cave.py, Python, 238 lines
navis/utils/cv.py, Python, 123 lines
navis/utils/decorators.py, Python, 774 lines
navis/utils/eval.py, Python, 356 lines
navis/utils/exceptions.py, Python, 25 lines
navis/utils/http.py, Python, 145 lines
navis/utils/iterables.py, Python, 135 lines
navis/utils/meshproc.py, Python, 271 lines
navis/utils/misc.py, Python, 576 lines
navis/utils/subclasses.py, Python, 202 lines
navis/utils/validate.py, Python, 147 lines
run_mypy.sh, Shell, 3 lines
scripts/gallery_index_hook.py, Python, 370 lines
scripts/gen_llms_txt.py, Python, 577 lines
scripts/gen_ref_pages.py, Python, 70 lines
scripts/gen_ref_pages2.py, Python, 196 lines
setup.py, Python, 76 lines
stubs/numpy/__init__.py, Python, 1 line
stubs/numpy/linalg.py, Python, 4 lines
stubs/pandas/__init__.py, Python, 1 line
tests/__init__.py, Python, 1 line
tests/conftest.py, Python, 216 lines
tests/fixtures/r_data/generate.py, Python, 41 lines
tests/test_align.py, Python, 196 lines
tests/test_angles.py, Python, 188 lines
tests/test_bil.py, Python, 241 lines
tests/test_caps.py, Python, 195 lines
tests/test_cast_neuron.py, Python, 141 lines
tests/test_cave_patch.py, Python, 211 lines
tests/test_collage.py, Python, 593 lines
tests/test_compute_backends.py, Python, 873 lines
tests/test_compute_cluster.py, Python, 291 lines
tests/test_compute_threads.py, Python, 325 lines
tests/test_connected_components.py, Python, 354 lines
tests/test_connecting_nodes.py, Python, 218 lines
tests/test_connectivity.py, Python, 138 lines
tests/test_deprecated_names.py, Python, 145 lines
tests/test_dotprops.py, Python, 310 lines
tests/test_downsample.py, Python, 378 lines
tests/test_error_messages.py, Python, 206 lines
tests/test_extra_edges.py, Python, 349 lines
tests/test_face_connectivity.py, Python, 409 lines
tests/test_flow_metrics.py, Python, 114 lines
tests/test_geodesic_clusters.py, Python, 223 lines
tests/test_geodesic_nearest.py, Python, 136 lines
tests/test_graph_contracts.py, Python, 639 lines
tests/test_graph_primitives.py, Python, 171 lines
tests/test_guess_change.py, Python, 187 lines
tests/test_heal.py, Python, 412 lines
tests/test_heal_mesh.py, Python, 283 lines
tests/test_interfaces_base.py, Python, 344 lines
tests/test_io.py, Python, 780 lines
tests/test_io_url.py, Python, 131 lines
tests/test_ivscc.py, Python, 401 lines
tests/test_links.py, Python, 1,511 lines
tests/test_llms_txt.py, Python, 223 lines
tests/test_masking.py, Python, 547 lines
tests/test_mesh_ops.py, Python, 611 lines
tests/test_mesh_process.py, Python, 292 lines
tests/test_ml.py, Python, 918 lines
tests/test_models/__init__.py, Python, 1 line
tests/test_models/test_network_models.py, Python, 72 lines
tests/test_morpho_contracts.py, Python, 251 lines
tests/test_nbl/__init__.py, Python, 1 line
tests/test_nbl/test_extract_matches.py, Python, 293 lines
tests/test_nbl/test_nblast_dispatch.py, Python, 259 lines
tests/test_nbl/test_nblast_knn.py, Python, 255 lines
tests/test_nbl/test_partition.py, Python, 131 lines
tests/test_nbl/test_smat.py, Python, 142 lines
tests/test_neuron_methods.py, Python, 467 lines
tests/test_neurons.py, Python, 337 lines
tests/test_node_label_sorting.py, Python, 109 lines
tests/test_parallel.py, Python, 288 lines
tests/test_pipeline.py, Python, 616 lines
tests/test_plotting.py, Python, 2,073 lines
tests/test_propagate_labels.py, Python, 106 lines
tests/test_quiet_logger.py, Python, 121 lines
tests/test_resample.py, Python, 487 lines
tests/test_sample.py, Python, 344 lines
tests/test_schema.py, Python, 613 lines
tests/test_sholl.py, Python, 206 lines
tests/test_smooth.py, Python, 343 lines
tests/test_soma.py, Python, 232 lines
tests/test_transform_backends.py, Python, 1,240 lines
tests/test_transforms.py, Python, 93 lines
tests/test_tutorials.py, Python, 113 lines
tests/test_voxel.py, Python, 1,754 lines
LICENSE, License, 674 lines
README.md, Text, 123 lines

At the source: github.com/navis-org/navis

aplbrain/dotmotif

License: Apache-2.0

State: the link answers, verified on 28 September 2026

Evidence: files inventoried

Commit: 125fdbca18c1c8ac7ed1b8d3f595669edc36574f, 23 September 2026

Languages: Python (27)

Size: 58 files, 27 scripts

Software Heritage: archived

Found in: the end of the paper

Holds: README, license file, environment (pyproject.toml, uv.lock), tests, continuous integration, documentation

Not found: CITATION.cff

Tools: NetworkX (13 files), pandas (3 files)

Availability: 1 check, the latest on 28 September 2026: the link answers

28 September 2026: the link answers

29 files

docs/examples/Counting Triangles in MICrONS v185.py, Python, 23 lines
docs/examples/Searching in neuPrint.py, Python, 17 lines
dotmotif/__init__.py, Python, 389 lines
dotmotif/executors/Executor.py, Python, 11 lines
dotmotif/executors/GrandIsoExecutor.py, Python, 122 lines
dotmotif/executors/Neo4jExecutor.py, Python, 480 lines
dotmotif/executors/NetworkXExecutor.py, Python, 458 lines
dotmotif/executors/NeuPrintExecutor.py, Python, 151 lines
dotmotif/executors/__init__.py, Python, 21 lines
dotmotif/executors/test_dm_cypher.py, Python, 270 lines
dotmotif/executors/test_grandisoexecutor.py, Python, 717 lines
dotmotif/executors/test_neo4jexecutor.py, Python, 48 lines
dotmotif/executors/test_networkxexecutor.py, Python, 736 lines
dotmotif/executors/test_neuprintexecutor.py, Python, 96 lines
dotmotif/ingest/__init__.py, Python, 207 lines
dotmotif/ingest/test_ingest.py, Python, 130 lines
dotmotif/parsers/__init__.py, Python, 11 lines
dotmotif/parsers/v2/__init__.py, Python, 631 lines
dotmotif/parsers/v2/test_v2_parser.py, Python, 613 lines
dotmotif/tests/__init__.py, Python, 1 line
dotmotif/tests/test_automorphism_exclusion.py, Python, 121 lines
dotmotif/tests/test_dm_flags.py, Python, 256 lines
dotmotif/tests/test_multigraphs.py, Python, 206 lines
dotmotif/tests/test_utils.py, Python, 87 lines
dotmotif/utils.py, Python, 130 lines
dotmotif/validators/__init__.py, Python, 182 lines
dotmotif/validators/test_dm_illegal_operators.py, Python, 19 lines
LICENSE, License, 174 lines
README.md, Text, 88 lines

At the source: github.com/aplbrain/dotmotif

google/neuroglancer

License: Apache-2.0

State: the link answers, verified on 27 October 2026

Evidence: files inventoried

Commit: bcecdb0cbb877191e20cfa358803f6e7547583c2, 26 October 2026

Languages: TypeScript (710), Python (133), C++ (23), JavaScript (20), Shell (11), C/C++ (6), Go (3), NEURON (1), Jupyter (1), Rust (1), C (1)

Size: 2,071 files, 910 scripts

Software Heritage: archived

Found in: the end of the paper

Holds: README, license file, environment (pyproject.toml, setup.py, uv.lock, docs/pyproject.toml, python/Dockerfile, src/mesh/draco/Dockerfile, src/sliceview/compresso/Dockerfile, src/sliceview/crackle/Dockerfile, src/sliceview/jxl/compile.Dockerfile, src/sliceview/jxl/optimize.Dockerfile, src/sliceview/png/Dockerfile), tests, continuous integration, documentation, 1 notebook

Not found: CITATION.cff

Tools: NumPy (67 files), Pillow (3 files), SciPy (3 files), pandas (1 file)

Availability: 2 checks, the latest on 27 October 2026: the link answers

27 October 2026: the link answers
27 September 2026: the link answers

912 files

build_tools/after-version-change.ts, TypeScript, 48 lines
build_tools/build-package.ts, TypeScript, 247 lines
build_tools/cli.ts, TypeScript, 266 lines
build_tools/postpack.ts, TypeScript, 9 lines
build_tools/rspack/configuration_with_define.js, JavaScript, 13 lines
build_tools/rspack/rspack_config_from_cli.js, JavaScript, 12 lines
build_tools/update-conditions.ts, TypeScript, 147 lines
build_tools/update-example-dependencies.ts, TypeScript, 30 lines
build_tools/vitest/build_fake_gcs_server.ts, TypeScript, 57 lines
build_tools/vitest/fake_ngauth_server.ts, TypeScript, 98 lines
build_tools/vitest/polyfill-browser-globals-in-node.ts, TypeScript, 16 lines
build_tools/vitest/python_tools.ts, TypeScript, 26 lines
build_tools/vitest/test_data_server.ts, TypeScript, 51 lines
build_tools/vitest/vitest-environment-jsdom-patched/index.ts, TypeScript, 50 lines
cors_webserver.py, Python, 62 lines
docs/concepts/data_source_coordinate_space.screenshot.py, Python, 59 lines
docs/concepts/data_source_coordinate_transform.screenshot.py, Python, 59 lines
docs/conf.py, Python, 344 lines
docs/generate_logo.py, Python, 122 lines
docs/update_intersphinx_inventories.py, Python, 42 lines
docs/user-guide/navigation_mouse_cross_section_translate.video.py, Python, 28 lines
docs/user-guide/navigation_mouse_cross_section_zoom.video.py, Python, 30 lines
eslint.config.js, JavaScript, 124 lines
examples/parcel/parcel-project-built/src/index.js, JavaScript, 3 lines
examples/parcel/parcel-project-source/src/index.js, JavaScript, 3 lines
examples/rsbuild/rsbuild-project-built/rsbuild.config.ts, TypeScript, 45 lines
examples/rsbuild/rsbuild-project-built/src/index.js, JavaScript, 3 lines
examples/rsbuild/rsbuild-project-source/rsbuild.config.ts, TypeScript, 60 lines
examples/rsbuild/rsbuild-project-source/src/index.js, JavaScript, 3 lines
examples/rspack/rspack-project-built/rspack.config.js, JavaScript, 46 lines
examples/rspack/rspack-project-built/src/index.js, JavaScript, 3 lines
examples/rspack/rspack-project-source/rspack.config.js, JavaScript, 61 lines
examples/rspack/rspack-project-source/src/index.js, JavaScript, 3 lines
examples/vite/vite-project-built/vite.config.ts, TypeScript, 47 lines
examples/vite/vite-project-source/vite.config.ts, TypeScript, 51 lines
examples/webpack/webpack-project-built/src/index.js, JavaScript, 3 lines
examples/webpack/webpack-project-built/webpack.config.js, JavaScript, 48 lines
examples/webpack/webpack-project-source/src/index.js, JavaScript, 3 lines
examples/webpack/webpack-project-source/webpack.config.js, JavaScript, 58 lines
ngauth_server/auth.go, Go, 521 lines
ngauth_server/bounded_access_token.go, Go, 95 lines
ngauth_server/go.mod, NEURON, 30 lines
ngauth_server/main.go, Go, 52 lines
noxfile.py, Python, 195 lines
playwright.config.ts, TypeScript, 22 lines
python/build_tools/cibuildwheel_linux_cache_setup.sh, Shell, 12 lines
python/copy_openmesh_deps.py, Python, 49 lines
python/examples/agglomeration_split_tool_csv_to_sqlite.py, Python, 83 lines
python/examples/example.py, Python, 70 lines
python/examples/example_action.py, Python, 30 lines
python/examples/example_add_cube.py, Python, 55 lines
python/examples/example_annotation_properties.py, Python, 66 lines
python/examples/example_cdf.py, Python, 58 lines
python/examples/example_coordinate_arrays.py, Python, 58 lines
python/examples/example_coordinate_transform.py, Python, 46 lines
python/examples/example_cross_section.py, Python, 40 lines
python/examples/example_dask.py, Python, 53 lines
python/examples/example_fixed_token.py, Python, 20 lines
python/examples/example_grid_layout.py, Python, 39 lines
python/examples/example_layer_side_panels.py, Python, 39 lines
python/examples/example_local_volume_coordinate_arrays.py, Python, 63 lines
python/examples/example_overlay.py, Python, 49 lines
python/examples/example_partial_viewport.py, Python, 87 lines
python/examples/example_precomputed_gcs.py, Python, 16 lines
python/examples/example_read_precomputed_annotations.py, Python, 23 lines
python/examples/example_row_layout.py, Python, 29 lines
python/examples/example_signed_int.py, Python, 45 lines
python/examples/example_single_mesh_layer.py, Python, 16 lines
python/examples/example_skeletons.py, Python, 76 lines
python/examples/example_toggle_visibility.py, Python, 51 lines
python/examples/example_tool.py, Python, 24 lines
python/examples/extend_segments_tool.py, Python, 290 lines
python/examples/flood_filling_simulation.py, Python, 250 lines
python/examples/interactive_inference.py, Python, 115 lines
python/examples/jupyter-notebook-demo.ipynb, Jupyter, 177 lines
python/examples/synaptic_partners.py, Python, 192 lines
python/examples/webdriver_example.py, Python, 32 lines
python/examples/write_annotations.py, Python, 207 lines
python/ext/src/_neuroglancer.cc, C++, 208 lines
python/ext/src/compress_segmentation.cc, C++, 232 lines
python/ext/src/compress_segmentation.h, C/C++, 173 lines
python/ext/src/compress_segmentation_test.cc, C++, 229 lines
python/ext/src/generate_marching_cubes_tables.py, Python, 72 lines
python/ext/src/mesh_objects.cc, C++, 147 lines
python/ext/src/mesh_objects.h, C/C++, 38 lines
python/ext/src/on_demand_object_mesh_generator.cc, C++, 226 lines
python/ext/src/on_demand_object_mesh_generator.h, C/C++, 61 lines
python/ext/src/openmesh_dependencies.cc, C++, 22 lines
python/ext/src/voxel_mesh_generator.cc, C++, 479 lines
python/ext/src/voxel_mesh_generator.h, C/C++, 246 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/ArrayKernel.cc, C++, 260 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/ArrayKernelT.cc, C++, 315 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/PolyConnectivity.cc, C++, 1,233 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/PolyMeshT.cc, C++, 458 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/TriConnectivity.cc, C++, 500 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Mesh/TriMeshT.cc, C++, 93 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/System/config.h, C/C++, 111 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/System/omstream.cc, C++, 107 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Utils/BaseProperty.cc, C++, 59 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Core/Utils/SingletonT.cc, C++, 85 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Tools/Decimater/BaseDecimaterT.cc, C++, 310 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Tools/Decimater/DecimaterT.cc, C++, 369 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Tools/Decimater/ModQuadricT.cc, C++, 159 lines
python/ext/third_party/openmesh/OpenMesh/src/OpenMesh/Tools/Decimater/Observer.cc, C++, 98 lines
python/neuroglancer/__init__.py, Python, 122 lines
python/neuroglancer/async_util.py, Python, 86 lines
python/neuroglancer/boss_credentials.py, Python, 55 lines
python/neuroglancer/chunks.py, Python, 39 lines
python/neuroglancer/cli.py, Python, 93 lines
python/neuroglancer/coordinate_space.py, Python, 320 lines
python/neuroglancer/credentials_provider.py, Python, 69 lines
python/neuroglancer/default_credentials_manager.py, Python, 56 lines
python/neuroglancer/downsample.py, Python, 42 lines
python/neuroglancer/downsample_scales.py, Python, 101 lines
python/neuroglancer/dvid_credentials.py, Python, 57 lines
python/neuroglancer/equivalence_map.py, Python, 193 lines
python/neuroglancer/futures.py, Python, 63 lines
python/neuroglancer/google_credentials.py, Python, 85 lines
python/neuroglancer/json_utils.py, Python, 59 lines
python/neuroglancer/json_wrappers.py, Python, 649 lines
python/neuroglancer/local_volume.py, Python, 386 lines
python/neuroglancer/random_token.py, Python, 21 lines
python/neuroglancer/read_precomputed_annotations.py, Python, 642 lines
python/neuroglancer/read_precomputed_segment_properties.py, Python, 94 lines
python/neuroglancer/read_precomputed_skeletons.py, Python, 102 lines
python/neuroglancer/screenshot.py, Python, 51 lines
python/neuroglancer/segment_colors.py, Python, 91 lines
python/neuroglancer/server.py, Python, 653 lines
python/neuroglancer/skeleton.py, Python, 101 lines
python/neuroglancer/static/__init__.py, Python, 102 lines
python/neuroglancer/static_file_server.py, Python, 64 lines
python/neuroglancer/test_util.py, Python, 13 lines
python/neuroglancer/test_utils.py, Python, 47 lines
python/neuroglancer/tool/__init__.py, Python, 1 line
python/neuroglancer/tool/agglomeration_split_tool.py, Python, 904 lines
python/neuroglancer/tool/cube.py, Python, 68 lines
python/neuroglancer/tool/edit_states.py, Python, 139 lines
python/neuroglancer/tool/filter_bodies.py, Python, 217 lines
python/neuroglancer/tool/mask_tool.py, Python, 319 lines
python/neuroglancer/tool/merge_tool.py, Python, 526 lines
python/neuroglancer/tool/save_meshes.py, Python, 92 lines
python/neuroglancer/tool/screenshot.py, Python, 815 lines
python/neuroglancer/tool/video_tool.py, Python, 664 lines
python/neuroglancer/trackable_state.py, Python, 217 lines
python/neuroglancer/url_state.py, Python, 158 lines
python/neuroglancer/viewer.py, Python, 56 lines
python/neuroglancer/viewer_base.py, Python, 507 lines
python/neuroglancer/viewer_config_state.py, Python, 369 lines
python/neuroglancer/viewer_state.py, Python, 1,996 lines
python/neuroglancer/webdriver.py, Python, 189 lines
python/neuroglancer/write_annotations.py, Python, 376 lines
python/tests/annotation_properties_test.py, Python, 113 lines
python/tests/annotation_tool_test.py, Python, 100 lines
python/tests/conftest.py, Python, 186 lines
python/tests/context_lost_test.py, Python, 59 lines
python/tests/display_dimensions_test.py, Python, 64 lines
python/tests/equivalence_map_test.py, Python, 78 lines
python/tests/fill_value_test.py, Python, 104 lines
python/tests/json_wrappers_test.py, Python, 59 lines
python/tests/linked_segment_group_test.py, Python, 64 lines
python/tests/local_volume_test.py, Python, 81 lines
python/tests/managed_layer_test.py, Python, 10 lines
python/tests/merge_tool_test.py, Python, 84 lines
python/tests/n5_test.py, Python, 74 lines
python/tests/on_demand_mesh_generator_test.py, Python, 56 lines
python/tests/picking_test.py, Python, 259 lines
python/tests/precomputed_test.py, Python, 93 lines
python/tests/screenshot_test.py, Python, 41 lines
python/tests/segment_colors_test.py, Python, 132 lines
python/tests/selected_values_test.py, Python, 97 lines
python/tests/shader_controls_test.py, Python, 186 lines
python/tests/skeleton_options_test.py, Python, 79 lines
python/tests/title_test.py, Python, 49 lines
python/tests/url_state_test.py, Python, 50 lines
python/tests/viewer_config_state_test.py, Python, 49 lines
python/tests/viewer_state_roundtrip_test.py, Python, 73 lines
python/tests/viewer_state_test.py, Python, 118 lines
python/tests/volume_rendering_test.py, Python, 172 lines
python/tests/write_annotations_test.py, Python, 151 lines
python/tests/zarr_test.py, Python, 669 lines
rspack.config.ts, TypeScript, 128 lines
setup.py, Python, 293 lines
src/annotation/annotation_layer_state.ts, TypeScript, 244 lines
src/annotation/backend.ts, TypeScript, 574 lines
src/annotation/base.ts, TypeScript, 170 lines
src/annotation/bounding_box.ts, TypeScript, 704 lines
src/annotation/ellipsoid.ts, TypeScript, 361 lines
src/annotation/frontend_source.browser_test.ts, TypeScript, 152 lines
src/annotation/frontend_source.ts, TypeScript, 1,058 lines
src/annotation/index.ts, TypeScript, 1,940 lines
src/annotation/line.ts, TypeScript, 300 lines
src/annotation/point.ts, TypeScript, 237 lines
src/annotation/polyline.ts, TypeScript, 342 lines
src/annotation/renderlayer.ts, TypeScript, 1,217 lines
src/annotation/selection.ts, TypeScript, 49 lines
src/annotation/type_handler.ts, TypeScript, 854 lines
src/async_computation.bundle.js, JavaScript, 8 lines
src/async_computation/decode_blosc.ts, TypeScript, 25 lines
src/async_computation/decode_blosc_request.ts, TypeScript, 22 lines
src/async_computation/decode_compresso.ts, TypeScript, 24 lines
src/async_computation/decode_compresso_request.ts, TypeScript, 22 lines
src/async_computation/decode_crackle.ts, TypeScript, 24 lines
src/async_computation/decode_crackle_request.ts, TypeScript, 21 lines
src/async_computation/decode_jpeg.ts, TypeScript, 88 lines
src/async_computation/decode_jpeg_request.ts, TypeScript, 29 lines
src/async_computation/decode_jxl.ts, TypeScript, 37 lines
src/async_computation/decode_jxl_request.ts, TypeScript, 27 lines
src/async_computation/decode_png.ts, TypeScript, 44 lines
src/async_computation/decode_png_request.ts, TypeScript, 36 lines
src/async_computation/decode_zstd.ts, TypeScript, 25 lines
src/async_computation/decode_zstd_node.ts, TypeScript, 30 lines
src/async_computation/decode_zstd_request.ts, TypeScript, 22 lines
src/async_computation/encode_compressed_segmentation.ts, TypeScript, 44 lines
src/async_computation/encode_compressed_segmentation_request.ts, TypeScript, 33 lines
src/async_computation/handler.ts, TypeScript, 58 lines
src/async_computation/index.ts, TypeScript, 33 lines
src/async_computation/obj_mesh.ts, TypeScript, 76 lines
src/async_computation/obj_mesh_request.ts, TypeScript, 25 lines
src/async_computation/request.ts, TypeScript, 124 lines
src/async_computation/vtk_mesh.ts, TypeScript, 35 lines
src/async_computation/vtk_mesh_request.ts, TypeScript, 25 lines
src/axes_lines.ts, TypeScript, 163 lines
src/chunk_manager/backend.ts, TypeScript, 1,391 lines
src/chunk_manager/base.ts, TypeScript, 121 lines
src/chunk_manager/frontend.ts, TypeScript, 533 lines
src/chunk_manager/generic_file_source.ts, TypeScript, 150 lines
src/chunk_worker.bundle.js, JavaScript, 14 lines
src/coordinate_transform.spec.ts, TypeScript, 517 lines
src/coordinate_transform.ts, TypeScript, 1,976 lines
src/credentials_provider/chunk_source_frontend.ts, TypeScript, 99 lines
src/credentials_provider/default_manager.ts, TypeScript, 40 lines
src/credentials_provider/http_request.ts, TypeScript, 85 lines
src/credentials_provider/index.ts, TypeScript, 207 lines
src/credentials_provider/interactive_credentials_provider.ts, TypeScript, 129 lines
src/credentials_provider/oauth2.ts, TypeScript, 94 lines
src/credentials_provider/shared.ts, TypeScript, 117 lines
src/credentials_provider/shared_common.ts, TypeScript, 20 lines
src/credentials_provider/shared_counterpart.ts, TypeScript, 131 lines
src/data_management_context.ts, TypeScript, 79 lines
src/data_panel_layout.browser_test.ts, TypeScript, 132 lines
src/data_panel_layout.ts, TypeScript, 1,023 lines
src/datasource/boss/api.ts, TypeScript, 63 lines
src/datasource/boss/async_computation.ts, TypeScript, 1 line
src/datasource/boss/backend.ts, TypeScript, 143 lines
src/datasource/boss/base.ts, TypeScript, 46 lines
src/datasource/boss/credentials_provider.ts, TypeScript, 172 lines
src/datasource/boss/frontend.ts, TypeScript, 980 lines
src/datasource/boss/register_credentials_provider.ts, TypeScript, 24 lines
src/datasource/boss/register_default.ts, TypeScript, 20 lines
src/datasource/brainmaps/api.ts, TypeScript, 108 lines
src/datasource/brainmaps/async_computation.ts, TypeScript, 1 line
src/datasource/brainmaps/backend.ts, TypeScript, 1,264 lines
src/datasource/brainmaps/base.ts, TypeScript, 123 lines
src/datasource/brainmaps/credentials_provider.ts, TypeScript, 37 lines
src/datasource/brainmaps/frontend.ts, TypeScript, 1,267 lines
src/datasource/brainmaps/register_credentials_provider.ts, TypeScript, 29 lines
src/datasource/brainmaps/register_default.ts, TypeScript, 36 lines
src/datasource/deepzoom/async_computation.ts, TypeScript, 2 lines
src/datasource/deepzoom/backend.ts, TypeScript, 151 lines
src/datasource/deepzoom/base.ts, TypeScript, 31 lines
src/datasource/deepzoom/frontend.ts, TypeScript, 345 lines
src/datasource/deepzoom/register_default.ts, TypeScript, 31 lines
src/datasource/default_provider.ts, TypeScript, 64 lines
src/datasource/dvid/api.ts, TypeScript, 102 lines
src/datasource/dvid/async_computation.ts, TypeScript, 1 line
src/datasource/dvid/backend.ts, TypeScript, 214 lines
src/datasource/dvid/base.ts, TypeScript, 44 lines
src/datasource/dvid/credentials_provider.ts, TypeScript, 86 lines
src/datasource/dvid/frontend.ts, TypeScript, 778 lines
src/datasource/dvid/register_credentials_provider.ts, TypeScript, 29 lines
src/datasource/dvid/register_default.ts, TypeScript, 20 lines
src/datasource/enabled_async_computation_modules.ts, TypeScript, 12 lines
src/datasource/enabled_backend_modules.ts, TypeScript, 14 lines
src/datasource/enabled_frontend_modules.ts, TypeScript, 18 lines
src/datasource/graphene/async_computation.ts, TypeScript, 1 line
src/datasource/graphene/backend.ts, TypeScript, 644 lines
src/datasource/graphene/base.ts, TypeScript, 174 lines
src/datasource/graphene/frontend.ts, TypeScript, 3,119 lines
src/datasource/graphene/register_default.ts, TypeScript, 23 lines
src/datasource/index.ts, TypeScript, 686 lines
src/datasource/kvstore_completions.ts, TypeScript, 218 lines
src/datasource/local.ts, TypeScript, 132 lines
src/datasource/n5/async_computation.ts, TypeScript, 2 lines
src/datasource/n5/backend.ts, TypeScript, 120 lines
src/datasource/n5/base.ts, TypeScript, 30 lines
src/datasource/n5/frontend.ts, TypeScript, 723 lines
src/datasource/n5/register_default.ts, TypeScript, 31 lines
src/datasource/nggraph/credentials_provider.ts, TypeScript, 100 lines
src/datasource/nggraph/frontend.ts, TypeScript, 798 lines
src/datasource/nggraph/register_default.ts, TypeScript, 20 lines
src/datasource/nifti/backend.ts, TypeScript, 316 lines
src/datasource/nifti/base.ts, TypeScript, 38 lines
src/datasource/nifti/frontend.ts, TypeScript, 260 lines
src/datasource/nifti/register_default.ts, TypeScript, 31 lines
src/datasource/obj/async_computation.ts, TypeScript, 1 line
src/datasource/obj/backend.ts, TypeScript, 45 lines
src/datasource/obj/register_default.ts, TypeScript, 26 lines
src/datasource/precomputed/async_computation.ts, TypeScript, 5 lines
src/datasource/precomputed/backend.ts, TypeScript, 1,095 lines
src/datasource/precomputed/base.ts, TypeScript, 118 lines
src/datasource/precomputed/frontend.ts, TypeScript, 1,426 lines
src/datasource/precomputed/register_default.ts, TypeScript, 30 lines
src/datasource/precomputed/sharded.ts, TypeScript, 312 lines
src/datasource/python/backend.ts, TypeScript, 137 lines
src/datasource/python/base.ts, TypeScript, 45 lines
src/datasource/python/frontend.browser_test.ts, TypeScript, 43 lines
src/datasource/python/frontend.ts, TypeScript, 671 lines
src/datasource/python/register_default.ts, TypeScript, 20 lines
src/datasource/render/async_computation.ts, TypeScript, 1 line
src/datasource/render/backend.ts, TypeScript, 155 lines
src/datasource/render/base.ts, TypeScript, 35 lines
src/datasource/render/frontend.ts, TypeScript, 846 lines
src/datasource/render/register_default.ts, TypeScript, 20 lines
src/datasource/state_share.ts, TypeScript, 163 lines
src/datasource/vtk/async_computation.ts, TypeScript, 1 line
src/datasource/vtk/backend.ts, TypeScript, 72 lines
src/datasource/vtk/parse.ts, TypeScript, 299 lines
src/datasource/vtk/register_default.ts, TypeScript, 26 lines
src/datasource/zarr/async_computation.ts, TypeScript, 2 lines
src/datasource/zarr/backend.ts, TypeScript, 100 lines
src/datasource/zarr/base.ts, TypeScript, 23 lines
src/datasource/zarr/codec/blosc/decode.ts, TypeScript, 35 lines
src/datasource/zarr/codec/blosc/resolve.ts, TypeScript, 30 lines
src/datasource/zarr/codec/bytes/decode.ts, TypeScript, 53 lines
src/datasource/zarr/codec/bytes/resolve.ts, TypeScript, 71 lines
src/datasource/zarr/codec/crc32c/decode.ts, TypeScript, 37 lines
src/datasource/zarr/codec/crc32c/resolve.ts, TypeScript, 36 lines
src/datasource/zarr/codec/decode.ts, TypeScript, 211 lines
src/datasource/zarr/codec/gzip/decode.ts, TypeScript, 36 lines
src/datasource/zarr/codec/gzip/resolve.ts, TypeScript, 39 lines
src/datasource/zarr/codec/index.ts, TypeScript, 66 lines
src/datasource/zarr/codec/resolve.ts, TypeScript, 226 lines
src/datasource/zarr/codec/sharding_indexed/decode.ts, TypeScript, 201 lines
src/datasource/zarr/codec/sharding_indexed/resolve.ts, TypeScript, 127 lines
src/datasource/zarr/codec/transpose/decode.ts, TypeScript, 36 lines
src/datasource/zarr/codec/transpose/resolve.ts, TypeScript, 113 lines
src/datasource/zarr/codec/zstd/decode.ts, TypeScript, 35 lines
src/datasource/zarr/codec/zstd/resolve.ts, TypeScript, 30 lines
src/datasource/zarr/frontend.ts, TypeScript, 627 lines
src/datasource/zarr/metadata/index.ts, TypeScript, 51 lines
src/datasource/zarr/metadata/parse.ts, TypeScript, 444 lines
src/datasource/zarr/metadata/parse_util.ts, TypeScript, 41 lines
src/datasource/zarr/ome.spec.ts, TypeScript, 1,419 lines
src/datasource/zarr/ome.ts, TypeScript, 881 lines
src/datasource/zarr/register_default.ts, TypeScript, 38 lines
src/display_context.ts, TypeScript, 718 lines
src/gpu_hash/hash_function.ts, TypeScript, 32 lines
src/gpu_hash/hash_table.benchmark.ts, TypeScript, 32 lines
src/gpu_hash/hash_table.spec.ts, TypeScript, 140 lines
src/gpu_hash/hash_table.ts, TypeScript, 456 lines
src/gpu_hash/shader.browser_test.ts, TypeScript, 188 lines
src/gpu_hash/shader.ts, TypeScript, 224 lines
src/help/input_event_bindings.ts, TypeScript, 268 lines
src/kvstore/auto_detect.ts, TypeScript, 281 lines
src/kvstore/backend.ts, TypeScript, 65 lines
src/kvstore/byte_range/file_handle.ts, TypeScript, 109 lines
src/kvstore/byte_range/index.ts, TypeScript, 75 lines
src/kvstore/byte_range/register.ts, TypeScript, 41 lines
src/kvstore/chunk_source_frontend.ts, TypeScript, 55 lines
src/kvstore/context.ts, TypeScript, 185 lines
src/kvstore/enabled_async_computation_modules.ts, TypeScript, 3 lines
src/kvstore/enabled_backend_modules.ts, TypeScript, 11 lines
src/kvstore/enabled_frontend_modules.ts, TypeScript, 13 lines
src/kvstore/frontend.ts, TypeScript, 50 lines
src/kvstore/gcs/index.ts, TypeScript, 150 lines
src/kvstore/gcs/register.ts, TypeScript, 45 lines
src/kvstore/gzip/file_handle.ts, TypeScript, 235 lines
src/kvstore/gzip/index.ts, TypeScript, 96 lines
src/kvstore/gzip/register.ts, TypeScript, 50 lines
src/kvstore/http/backend.ts, TypeScript, 26 lines
src/kvstore/http/common.ts, TypeScript, 115 lines
src/kvstore/http/frontend.ts, TypeScript, 61 lines
src/kvstore/http/html_directory_listing.ts, TypeScript, 128 lines
src/kvstore/http/read.ts, TypeScript, 292 lines
src/kvstore/http/register_backend.ts, TypeScript, 21 lines
src/kvstore/http/register_frontend.ts, TypeScript, 21 lines
src/kvstore/icechunk/async_computation.ts, TypeScript, 1 line
src/kvstore/icechunk/auto_detect.ts, TypeScript, 37 lines
src/kvstore/icechunk/backend.ts, TypeScript, 112 lines
src/kvstore/icechunk/complete_url.ts, TypeScript, 100 lines
src/kvstore/icechunk/crockford_base32.ts, TypeScript, 61 lines
src/kvstore/icechunk/decode_utils.ts, TypeScript, 217 lines
src/kvstore/icechunk/frontend.ts, TypeScript, 42 lines
src/kvstore/icechunk/list.ts, TypeScript, 77 lines
src/kvstore/icechunk/manifest.ts, TypeScript, 102 lines
src/kvstore/icechunk/metadata_cache.ts, TypeScript, 249 lines
src/kvstore/icechunk/read.ts, TypeScript, 216 lines
src/kvstore/icechunk/ref.ts, TypeScript, 48 lines
src/kvstore/icechunk/register_backend.ts, TypeScript, 46 lines
src/kvstore/icechunk/register_frontend.ts, TypeScript, 59 lines
src/kvstore/icechunk/snapshot.ts, TypeScript, 261 lines
src/kvstore/icechunk/url.ts, TypeScript, 98 lines
src/kvstore/index.ts, TypeScript, 276 lines
src/kvstore/middleauth/common.ts, TypeScript, 90 lines
src/kvstore/middleauth/credentials_provider.ts, TypeScript, 218 lines
src/kvstore/middleauth/register_backend.ts, TypeScript, 21 lines
src/kvstore/middleauth/register_credentials_provider.ts, TypeScript, 33 lines
src/kvstore/middleauth/register_frontend.ts, TypeScript, 21 lines
src/kvstore/ngauth/credentials_provider.ts, TypeScript, 182 lines
src/kvstore/ngauth/register.ts, TypeScript, 84 lines
src/kvstore/ngauth/register_credentials_provider.ts, TypeScript, 43 lines
src/kvstore/ocdbt/async_computation.ts, TypeScript, 1 line
src/kvstore/ocdbt/auto_detect.ts, TypeScript, 27 lines
src/kvstore/ocdbt/backend.ts, TypeScript, 124 lines
src/kvstore/ocdbt/btree.ts, TypeScript, 474 lines
src/kvstore/ocdbt/complete_url.ts, TypeScript, 153 lines
src/kvstore/ocdbt/decode_utils.ts, TypeScript, 234 lines
src/kvstore/ocdbt/frontend.ts, TypeScript, 42 lines
src/kvstore/ocdbt/indirect_data_reference.ts, TypeScript, 192 lines
src/kvstore/ocdbt/key.ts, TypeScript, 101 lines
src/kvstore/ocdbt/list.ts, TypeScript, 232 lines
src/kvstore/ocdbt/list_versions.ts, TypeScript, 167 lines
src/kvstore/ocdbt/manifest.ts, TypeScript, 275 lines
src/kvstore/ocdbt/metadata_cache.ts, TypeScript, 191 lines
src/kvstore/ocdbt/read.ts, TypeScript, 154 lines
src/kvstore/ocdbt/read_version.ts, TypeScript, 330 lines
src/kvstore/ocdbt/register_backend.ts, TypeScript, 46 lines
src/kvstore/ocdbt/register_frontend.ts, TypeScript, 59 lines
src/kvstore/ocdbt/url.ts, TypeScript, 56 lines
src/kvstore/ocdbt/version_specifier.ts, TypeScript, 188 lines
src/kvstore/ocdbt/version_tree.ts, TypeScript, 621 lines
src/kvstore/proxy.ts, TypeScript, 250 lines
src/kvstore/register.ts, TypeScript, 79 lines
src/kvstore/s3/backend.ts, TypeScript, 26 lines
src/kvstore/s3/common.ts, TypeScript, 175 lines
src/kvstore/s3/frontend.ts, TypeScript, 52 lines
src/kvstore/s3/list.ts, TypeScript, 241 lines
src/kvstore/s3/register_backend.ts, TypeScript, 21 lines
src/kvstore/s3/register_frontend.ts, TypeScript, 21 lines
src/kvstore/shared_common.ts, TypeScript, 22 lines
src/kvstore/url.ts, TypeScript, 221 lines
src/kvstore/zip/auto_detect.ts, TypeScript, 46 lines
src/kvstore/zip/backend.ts, TypeScript, 256 lines
src/kvstore/zip/frontend.ts, TypeScript, 40 lines
src/kvstore/zip/metadata.ts, TypeScript, 658 lines
src/kvstore/zip/register_backend.ts, TypeScript, 43 lines
src/kvstore/zip/register_frontend.ts, TypeScript, 46 lines
src/layer/annotation/index.ts, TypeScript, 983 lines
src/layer/annotation/property_tools.spec.ts, TypeScript, 94 lines
src/layer/annotation/property_tools.ts, TypeScript, 669 lines
src/layer/annotation/shader_ui_property_list.ts, TypeScript, 49 lines
src/layer/annotation/tool_state.ts, TypeScript, 56 lines
src/layer/enabled_frontend_modules.ts, TypeScript, 5 lines
src/layer/image/index.ts, TypeScript, 614 lines
src/layer/index.ts, TypeScript, 2,545 lines
src/layer/layer_data_source.ts, TypeScript, 501 lines
src/layer/multi_channel_setup.ts, TypeScript, 430 lines
src/layer/segmentation/index.ts, TypeScript, 1,416 lines
src/layer/segmentation/json_keys.ts, TypeScript, 28 lines
src/layer/segmentation/layer_controls.ts, TypeScript, 156 lines
src/layer/single_mesh/index.ts, TypeScript, 281 lines
src/layer_group_viewer.ts, TypeScript, 643 lines
src/layer_groups_layout.ts, TypeScript, 825 lines
src/layout.ts, TypeScript, 51 lines
src/main.bundle.js, JavaScript, 5 lines
src/main.ts, TypeScript, 23 lines
src/main_module.ts, TypeScript, 4 lines
src/main_python.ts, TypeScript, 239 lines
src/mesh/backend.ts, TypeScript, 894 lines
src/mesh/base.ts, TypeScript, 51 lines
src/mesh/draco/build.sh, Shell, 13 lines
src/mesh/draco/build_wasm.py, Python, 138 lines
src/mesh/draco/draco_overlay/draco/draco_features.h, C/C++, 11 lines
src/mesh/draco/generate_first_bit_table.py, Python, 28 lines
src/mesh/draco/index.ts, TypeScript, 130 lines
src/mesh/draco/neuroglancer_draco.cc, C++, 158 lines
src/mesh/draco/stub.js, JavaScript, 6 lines
src/mesh/frontend.ts, TypeScript, 1,170 lines
src/mesh/multiscale.spec.ts, TypeScript, 1,048 lines
src/mesh/multiscale.ts, TypeScript, 403 lines
src/mesh/triangle_strips.spec.ts, TypeScript, 321 lines
src/mesh/triangle_strips.ts, TypeScript, 349 lines
src/navigation_state.ts, TypeScript, 2,377 lines
src/object_picking.ts, TypeScript, 131 lines
src/overlay.ts, TypeScript, 119 lines
src/perspective_view/backend.ts, TypeScript, 50 lines
src/perspective_view/base.ts, TypeScript, 17 lines
src/perspective_view/panel.ts, TypeScript, 1,495 lines
src/perspective_view/render_layer.ts, TypeScript, 124 lines
src/projection_parameters.ts, TypeScript, 78 lines
src/python_integration/api.ts, TypeScript, 257 lines
src/python_integration/credentials_provider.ts, TypeScript, 99 lines
src/python_integration/event_action_map.ts, TypeScript, 52 lines
src/python_integration/prefetch.ts, TypeScript, 119 lines
src/python_integration/remote_actions.ts, TypeScript, 87 lines
src/python_integration/remote_status_messages.ts, TypeScript, 96 lines
src/python_integration/screenshots.ts, TypeScript, 235 lines
src/python_integration/volume.ts, TypeScript, 598 lines
src/render_coordinate_transform.ts, TypeScript, 873 lines
src/render_layer_backend.ts, TypeScript, 104 lines
src/render_layer_common.ts, TypeScript, 21 lines
src/render_scale_statistics.ts, TypeScript, 161 lines
src/rendered_data_panel.ts, TypeScript, 852 lines
src/rendered_data_panel_picking.spec.ts, TypeScript, 84 lines
src/rendered_data_panel_picking.ts, TypeScript, 163 lines
src/renderlayer.ts, TypeScript, 290 lines
src/segment_color.browser_test.ts, TypeScript, 67 lines
src/segment_color.ts, TypeScript, 181 lines
src/segmentation_display_state/backend.ts, TypeScript, 100 lines
src/segmentation_display_state/base.ts, TypeScript, 137 lines
src/segmentation_display_state/frontend.ts, TypeScript, 1,178 lines
src/segmentation_display_state/property_map.spec.ts, TypeScript, 416 lines
src/segmentation_display_state/property_map.ts, TypeScript, 1,339 lines
src/segmentation_graph/local.spec.ts, TypeScript, 88 lines
src/segmentation_graph/local.ts, TypeScript, 304 lines
src/segmentation_graph/segment_id.ts, TypeScript, 29 lines
src/segmentation_graph/source.ts, TypeScript, 146 lines
src/shared_disjoint_sets.ts, TypeScript, 194 lines
src/shared_watchable_value.ts, TypeScript, 109 lines
src/single_mesh/backend.ts, TypeScript, 257 lines
src/single_mesh/base.ts, TypeScript, 56 lines
src/single_mesh/frontend.browser_test.ts, TypeScript, 97 lines
src/single_mesh/frontend.ts, TypeScript, 732 lines
src/skeleton/backend.ts, TypeScript, 202 lines
src/skeleton/base.ts, TypeScript, 24 lines
src/skeleton/decode_precomputed_skeleton.ts, TypeScript, 67 lines
src/skeleton/decode_swc_skeleton.spec.ts, TypeScript, 34 lines
src/skeleton/decode_swc_skeleton.ts, TypeScript, 113 lines
src/skeleton/frontend.ts, TypeScript, 914 lines
src/sliceview/backend.ts, TypeScript, 583 lines
src/sliceview/backend_chunk_decoders/bossNpz.ts, TypeScript, 65 lines
src/sliceview/backend_chunk_decoders/compressed_segmentation.ts, TypeScript, 26 lines
src/sliceview/backend_chunk_decoders/compresso.ts, TypeScript, 35 lines
src/sliceview/backend_chunk_decoders/crackle.ts, TypeScript, 35 lines
src/sliceview/backend_chunk_decoders/index.ts, TypeScript, 27 lines
src/sliceview/backend_chunk_decoders/jpeg.ts, TypeScript, 40 lines
src/sliceview/backend_chunk_decoders/jxl.ts, TypeScript, 38 lines
src/sliceview/backend_chunk_decoders/ndstoreNpz.ts, TypeScript, 59 lines
src/sliceview/backend_chunk_decoders/png.ts, TypeScript, 44 lines
src/sliceview/backend_chunk_decoders/postprocess.ts, TypeScript, 73 lines
src/sliceview/backend_chunk_decoders/raw.ts, TypeScript, 52 lines
src/sliceview/base.spec.ts, TypeScript, 202 lines
src/sliceview/base.ts, TypeScript, 1,065 lines
src/sliceview/bounding_box_shader_helper.ts, TypeScript, 442 lines
src/sliceview/chunk_format_handlers.ts, TypeScript, 18 lines
src/sliceview/chunk_format_testing.ts, TypeScript, 147 lines
src/sliceview/chunk_layout.ts, TypeScript, 93 lines
src/sliceview/compressed_segmentation/chunk_format.browser_test.ts, TypeScript, 148 lines
src/sliceview/compressed_segmentation/chunk_format.ts, TypeScript, 426 lines
src/sliceview/compressed_segmentation/decode_common.ts, TypeScript, 256 lines
src/sliceview/compressed_segmentation/decode_uint32.ts, TypeScript, 28 lines
src/sliceview/compressed_segmentation/decode_uint64.ts, TypeScript, 28 lines
src/sliceview/compressed_segmentation/encode.browser_benchmark.ts, TypeScript, 68 lines
src/sliceview/compressed_segmentation/encode_common.ts, TypeScript, 340 lines
src/sliceview/compressed_segmentation/encode_uint32.spec.ts, TypeScript, 360 lines
src/sliceview/compressed_segmentation/encode_uint32.ts, TypeScript, 29 lines
src/sliceview/compressed_segmentation/encode_uint64.spec.ts, TypeScript, 373 lines
src/sliceview/compressed_segmentation/encode_uint64.ts, TypeScript, 30 lines
src/sliceview/compressed_segmentation/test_util.ts, TypeScript, 34 lines
src/sliceview/compresso/build.sh, Shell, 13 lines
src/sliceview/compresso/build_wasm.sh, Shell, 23 lines
src/sliceview/compresso/compresso_wasm.cc, C++, 33 lines
src/sliceview/compresso/index.ts, TypeScript, 124 lines
src/sliceview/crackle/build.sh, Shell, 13 lines
src/sliceview/crackle/build_wasm.sh, Shell, 23 lines
src/sliceview/crackle/crackle_wasm.cc, C++, 30 lines
src/sliceview/crackle/index.ts, TypeScript, 159 lines
src/sliceview/frontend.ts, TypeScript, 1,119 lines
src/sliceview/jxl/build.sh, Shell, 22 lines
src/sliceview/jxl/build_wasm.sh, Shell, 6 lines
src/sliceview/jxl/index.ts, TypeScript, 166 lines
src/sliceview/jxl/optimize_wasm.sh, Shell, 4 lines
src/sliceview/jxl/src/lib.rs, Rust, 109 lines
src/sliceview/panel.ts, TypeScript, 568 lines
src/sliceview/png/build.sh, Shell, 13 lines
src/sliceview/png/build_wasm.sh, Shell, 30 lines
src/sliceview/png/index.ts, TypeScript, 258 lines
src/sliceview/png/png_wasm.c, C, 73 lines
src/sliceview/renderlayer.ts, TypeScript, 295 lines
src/sliceview/single_texture_chunk_format.ts, TypeScript, 157 lines
src/sliceview/uncompressed_chunk_format.browser_test.ts, TypeScript, 88 lines
src/sliceview/uncompressed_chunk_format.ts, TypeScript, 417 lines
src/sliceview/volume/backend.ts, TypeScript, 159 lines
src/sliceview/volume/base.ts, TypeScript, 312 lines
src/sliceview/volume/frontend.ts, TypeScript, 292 lines
src/sliceview/volume/image_renderlayer.ts, TypeScript, 161 lines
src/sliceview/volume/renderlayer.ts, TypeScript, 729 lines
src/sliceview/volume/segmentation_renderlayer.ts, TypeScript, 460 lines
src/sliceview/wire_frame.ts, TypeScript, 212 lines
src/status.ts, TypeScript, 231 lines
src/third_party/codemirror-glsl.ts, TypeScript, 260 lines
src/third_party/jpgjs/jpg.d.ts, TypeScript, 31 lines
src/third_party/jpgjs/jpg.js, JavaScript, 5,255 lines
src/trackable_alpha.ts, TypeScript, 29 lines
src/trackable_blend.ts, TypeScript, 43 lines
src/trackable_boolean.ts, TypeScript, 137 lines
src/trackable_finite_float.ts, TypeScript, 29 lines
src/trackable_value.ts, TypeScript, 542 lines
src/trackable_vec3.ts, TypeScript, 58 lines
src/ui/annotation_properties.ts, TypeScript, 204 lines
src/ui/annotation_schema_tab.ts, TypeScript, 1,902 lines
src/ui/annotations.ts, TypeScript, 2,940 lines
src/ui/bounded_number_input.spec.ts, TypeScript, 263 lines
src/ui/bounded_number_input.ts, TypeScript, 100 lines
src/ui/command.spec.ts, TypeScript, 37 lines
src/ui/command.ts, TypeScript, 156 lines
src/ui/command_catalog.spec.ts, TypeScript, 355 lines
src/ui/command_catalog.ts, TypeScript, 513 lines
src/ui/command_palette.ts, TypeScript, 356 lines
src/ui/command_registry.spec.ts, TypeScript, 132 lines
src/ui/command_registry.ts, TypeScript, 94 lines
src/ui/context_menu.ts, TypeScript, 150 lines
src/ui/copy_position.browser_test.ts, TypeScript, 55 lines
src/ui/default_clipboard_handling.spec.ts, TypeScript, 55 lines
src/ui/default_clipboard_handling.ts, TypeScript, 89 lines
src/ui/default_commands.spec.ts, TypeScript, 125 lines
src/ui/default_commands.ts, TypeScript, 271 lines
src/ui/default_input_event_bindings.ts, TypeScript, 206 lines
src/ui/default_viewer.ts, TypeScript, 29 lines
src/ui/default_viewer_setup.ts, TypeScript, 75 lines
src/ui/dialog_bindings.browser_test.ts, TypeScript, 79 lines
src/ui/disable_default_actions.ts, TypeScript, 40 lines
src/ui/drag_and_drop.ts, TypeScript, 110 lines
src/ui/layer_bar.ts, TypeScript, 523 lines
src/ui/layer_data_sources_tab.ts, TypeScript, 597 lines
src/ui/layer_drag_and_drop.ts, TypeScript, 595 lines
src/ui/layer_list_panel.ts, TypeScript, 488 lines
src/ui/layer_side_panel.ts, TypeScript, 483 lines
src/ui/layer_side_panel_state.ts, TypeScript, 363 lines
src/ui/minimal_viewer.ts, TypeScript, 43 lines
src/ui/position_drag_and_drop.ts, TypeScript, 96 lines
src/ui/screenshot_menu.ts, TypeScript, 931 lines
src/ui/segment_list.ts, TypeScript, 1,657 lines
src/ui/segment_select_tools.ts, TypeScript, 186 lines
src/ui/segment_split_merge_tools.ts, TypeScript, 532 lines
src/ui/segmentation_display_options_tab.ts, TypeScript, 144 lines
src/ui/selection_details.ts, TypeScript, 217 lines
src/ui/side_panel.ts, TypeScript, 781 lines
src/ui/side_panel_location.ts, TypeScript, 182 lines
src/ui/state_editor.ts, TypeScript, 173 lines
src/ui/statistics.ts, TypeScript, 409 lines
src/ui/title.ts, TypeScript, 36 lines
src/ui/tool.ts, TypeScript, 1,018 lines
src/ui/tool_drag_and_drop.ts, TypeScript, 42 lines
src/ui/tool_palette.ts, TypeScript, 1,344 lines
src/ui/tool_query.spec.ts, TypeScript, 322 lines
src/ui/tool_query.ts, TypeScript, 353 lines
src/ui/url_hash_binding.ts, TypeScript, 174 lines
src/ui/viewer_settings.ts, TypeScript, 148 lines
src/uint64_map.spec.ts, TypeScript, 125 lines
src/uint64_map.ts, TypeScript, 138 lines
src/uint64_ordered_set.ts, TypeScript, 117 lines
src/uint64_set.spec.ts, TypeScript, 101 lines
src/uint64_set.ts, TypeScript, 176 lines
src/util/abort.spec.ts, TypeScript, 91 lines
src/util/abort.ts, TypeScript, 137 lines
src/util/affine.spec.ts, TypeScript, 206 lines
src/util/affine.ts, TypeScript, 66 lines
src/util/animation_frame_debounce.ts, TypeScript, 64 lines
src/util/array.spec.ts, TypeScript, 307 lines
src/util/array.ts, TypeScript, 652 lines
src/util/automatic_focus.ts, TypeScript, 123 lines
src/util/base64.ts, TypeScript, 27 lines
src/util/bigint.ts, TypeScript, 62 lines
src/util/clipboard.spec.ts, TypeScript, 91 lines
src/util/clipboard.ts, TypeScript, 59 lines
src/util/color.browser_test.ts, TypeScript, 94 lines
src/util/color.ts, TypeScript, 249 lines
src/util/colorspace.ts, TypeScript, 106 lines
src/util/compare.ts, TypeScript, 22 lines
src/util/completion.ts, TypeScript, 181 lines
src/util/data_type.ts, TypeScript, 83 lines
src/util/disjoint_sets.spec.ts, TypeScript, 101 lines
src/util/disjoint_sets.ts, TypeScript, 241 lines
src/util/disposable.ts, TypeScript, 143 lines
src/util/dom.ts, TypeScript, 84 lines
src/util/drag_and_drop.spec.ts, TypeScript, 80 lines
src/util/drag_and_drop.ts, TypeScript, 282 lines
src/util/dropdown.ts, TypeScript, 112 lines
src/util/empirical_cdf.browser_test.ts, TypeScript, 256 lines
src/util/empirical_cdf.ts, TypeScript, 198 lines
src/util/endian.spec.ts, TypeScript, 34 lines
src/util/endian.ts, TypeScript, 145 lines
src/util/erf.ts, TypeScript, 39 lines
src/util/error.ts, TypeScript, 46 lines
src/util/event_action_map.spec.ts, TypeScript, 91 lines
src/util/event_action_map.ts, TypeScript, 548 lines
src/util/false.ts, TypeScript, 3 lines
src/util/file_download.ts, TypeScript, 27 lines
src/util/float.ts, TypeScript, 78 lines
src/util/float32_to_string.spec.ts, TypeScript, 70 lines
src/util/float32_to_string.ts, TypeScript, 38 lines
src/util/framerate.spec.ts, TypeScript, 128 lines
src/util/framerate.ts, TypeScript, 338 lines
src/util/geom.spec.ts, TypeScript, 201 lines
src/util/geom.ts, TypeScript, 524 lines
src/util/google_oauth2.ts, TypeScript, 263 lines
src/util/google_tag_manager.ts, TypeScript, 16 lines
src/util/gzip.ts, TypeScript, 71 lines
src/util/hash.ts, TypeScript, 149 lines
src/util/hex.ts, TypeScript, 35 lines
src/util/hierarchical_map.ts, TypeScript, 202 lines
src/util/http_request.ts, TypeScript, 146 lines
src/util/json.spec.ts, TypeScript, 87 lines
src/util/json.ts, TypeScript, 797 lines
src/util/keyboard_bindings.ts, TypeScript, 171 lines
src/util/leb128.spec.ts, TypeScript, 46 lines
src/util/leb128.ts, TypeScript, 52 lines
src/util/lerp.ts, TypeScript, 339 lines
src/util/linked_list.ts, TypeScript, 91 lines
src/util/longest_common_prefix.ts, TypeScript, 44 lines
src/util/matrix.spec.ts, TypeScript, 381 lines
src/util/matrix.ts, TypeScript, 360 lines
src/util/memoize.spec.ts, TypeScript, 207 lines
src/util/memoize.ts, TypeScript, 182 lines
src/util/message_list.ts, TypeScript, 80 lines
src/util/mixin.ts, TypeScript, 27 lines
src/util/mouse_bindings.ts, TypeScript, 99 lines
src/util/mouse_drag.ts, TypeScript, 55 lines
src/util/npy.spec.ts, TypeScript, 68 lines
src/util/npy.ts, TypeScript, 101 lines
src/util/number.ts, TypeScript, 22 lines
src/util/number_to_string.spec.ts, TypeScript, 28 lines
src/util/number_to_string.ts, TypeScript, 26 lines
src/util/numpy_dtype.ts, TypeScript, 81 lines
src/util/object_id.ts, TypeScript, 32 lines
src/util/pairing_heap.ts, TypeScript, 200 lines
src/util/platform.spec.ts, TypeScript, 57 lines
src/util/platform.ts, TypeScript, 42 lines
src/util/polyfills.ts, TypeScript, 2 lines
src/util/progress_listener.ts, TypeScript, 177 lines
src/util/random.ts, TypeScript, 52 lines
src/util/ranked_matches.spec.ts, TypeScript, 61 lines
src/util/ranked_matches.ts, TypeScript, 32 lines
src/util/screenshot_manager.ts, TypeScript, 462 lines
src/util/scroll_into_view.ts, TypeScript, 39 lines
src/util/si_units.spec.ts, TypeScript, 79 lines
src/util/si_units.ts, TypeScript, 194 lines
src/util/signal.spec.ts, TypeScript, 132 lines
src/util/signal.ts, TypeScript, 144 lines
src/util/signal_binding_updater.ts, TypeScript, 35 lines
src/util/spatial_units.ts, TypeScript, 85 lines
src/util/string.ts, TypeScript, 19 lines
src/util/touch_bindings.ts, TypeScript, 365 lines
src/util/trackable.ts, TypeScript, 201 lines
src/util/trackable_enum.ts, TypeScript, 53 lines
src/util/trackable_screenshot_mode.ts, TypeScript, 30 lines
src/util/true.ts, TypeScript, 3 lines
src/util/vector.ts, TypeScript, 131 lines
src/util/velocity_estimation.ts, TypeScript, 102 lines
src/util/viewer_resolution_stats.ts, TypeScript, 480 lines
src/util/watchable_map.ts, TypeScript, 109 lines
src/util/wheel_zoom.ts, TypeScript, 36 lines
src/util/zorder.spec.ts, TypeScript, 68 lines
src/util/zorder.ts, TypeScript, 136 lines
src/viewer.browser_test.ts, TypeScript, 84 lines
src/viewer.ts, TypeScript, 1,419 lines
src/viewer_state.ts, TypeScript, 39 lines
src/visibility_priority/backend.ts, TypeScript, 70 lines
src/visibility_priority/frontend.ts, TypeScript, 107 lines
src/volume_rendering/backend.ts, TypeScript, 205 lines
src/volume_rendering/base.ts, TypeScript, 197 lines
src/volume_rendering/trackable_volume_rendering_mode.ts, TypeScript, 26 lines
src/volume_rendering/volume_render_layer.browser_test.ts, TypeScript, 74 lines
src/volume_rendering/volume_render_layer.ts, TypeScript, 1,392 lines
src/webgl/bounding_box.ts, TypeScript, 112 lines
src/webgl/buffer.ts, TypeScript, 131 lines
src/webgl/circles.ts, TypeScript, 108 lines
src/webgl/colormaps.ts, TypeScript, 42 lines
src/webgl/context.ts, TypeScript, 75 lines
src/webgl/dynamic_shader.ts, TypeScript, 278 lines
src/webgl/ellipse.ts, TypeScript, 195 lines
src/webgl/empirical_cdf.ts, TypeScript, 239 lines
src/webgl/index_emulation.browser_test.ts, TypeScript, 57 lines
src/webgl/index_emulation.ts, TypeScript, 139 lines
src/webgl/lerp.browser_test.ts, TypeScript, 355 lines
src/webgl/lerp.ts, TypeScript, 392 lines
src/webgl/lines.ts, TypeScript, 154 lines
src/webgl/offscreen.ts, TypeScript, 412 lines
src/webgl/quad.ts, TypeScript, 51 lines
src/webgl/rectangle_grid_buffer.spec.ts, TypeScript, 41 lines
src/webgl/rectangle_grid_buffer.ts, TypeScript, 80 lines
src/webgl/shader.ts, TypeScript, 687 lines
src/webgl/shader_control_reachability.ts, TypeScript, 99 lines
src/webgl/shader_lib.browser_test.ts, TypeScript, 206 lines
src/webgl/shader_lib.ts, TypeScript, 642 lines
src/webgl/shader_source_string_preprocessing.ts, TypeScript, 45 lines
src/webgl/shader_testing.browser_test.ts, TypeScript, 118 lines
src/webgl/shader_testing.ts, TypeScript, 375 lines
src/webgl/shader_ui_controls.browser_test.ts, TypeScript, 1,548 lines
src/webgl/shader_ui_controls.ts, TypeScript, 1,975 lines
src/webgl/spheres.ts, TypeScript, 140 lines
src/webgl/square_corners_buffer.ts, TypeScript, 109 lines
src/webgl/testing.ts, TypeScript, 36 lines
src/webgl/texture.ts, TypeScript, 151 lines
src/webgl/texture_access.browser_test.ts, TypeScript, 101 lines
src/webgl/texture_access.ts, TypeScript, 614 lines
src/webgl/trivial_shaders.ts, TypeScript, 95 lines
src/webgl/vertex_id.ts, TypeScript, 80 lines
src/widget/add_button.ts, TypeScript, 23 lines
src/widget/annotation_tool_status.ts, TypeScript, 150 lines
src/widget/associate_label.ts, TypeScript, 27 lines
src/widget/channel_dimensions_widget.ts, TypeScript, 258 lines
src/widget/checkbox_icon.ts, TypeScript, 59 lines
src/widget/close_button.ts, TypeScript, 23 lines
src/widget/color.ts, TypeScript, 68 lines
src/widget/coordinate_transform.ts, TypeScript, 1,208 lines
src/widget/copy_button.ts, TypeScript, 28 lines
src/widget/delete_button.ts, TypeScript, 26 lines
src/widget/dependent_view_widget.ts, TypeScript, 92 lines
src/widget/display_dimensions_widget.ts, TypeScript, 865 lines
src/widget/enum_widget.ts, TypeScript, 70 lines
src/widget/eye_button.ts, TypeScript, 27 lines
src/widget/filter_button.ts, TypeScript, 23 lines
src/widget/help_button.ts, TypeScript, 22 lines
src/widget/hidden_submit_button.ts, TypeScript, 35 lines
src/widget/icon.ts, TypeScript, 70 lines
src/widget/invlerp.ts, TypeScript, 936 lines
src/widget/invlerp_range_finder.ts, TypeScript, 249 lines
src/widget/layer_control.ts, TypeScript, 246 lines
src/widget/layer_control_channel_invlerp.ts, TypeScript, 106 lines
src/widget/layer_control_checkbox.ts, TypeScript, 37 lines
src/widget/layer_control_color.ts, TypeScript, 53 lines
src/widget/layer_control_enum.ts, TypeScript, 49 lines
src/widget/layer_control_property_invlerp.ts, TypeScript, 134 lines
src/widget/layer_control_range.ts, TypeScript, 53 lines
src/widget/layer_control_select.spec.ts, TypeScript, 79 lines
src/widget/layer_control_select.ts, TypeScript, 97 lines
src/widget/layer_reference.ts, TypeScript, 62 lines
src/widget/layer_type_indicator.ts, TypeScript, 39 lines
src/widget/linked_layer.ts, TypeScript, 129 lines
src/widget/maximize_button.ts, TypeScript, 23 lines
src/widget/move_to_button.ts, TypeScript, 22 lines
src/widget/multiline_autocomplete.ts, TypeScript, 893 lines
src/widget/number_input_widget.ts, TypeScript, 79 lines
src/widget/position_plot.ts, TypeScript, 410 lines
src/widget/position_widget.ts, TypeScript, 1,726 lines
src/widget/progress_listener.ts, TypeScript, 47 lines
src/widget/range.ts, TypeScript, 93 lines
src/widget/render_scale_widget.ts, TypeScript, 438 lines
src/widget/scale_bar.ts, TypeScript, 444 lines
src/widget/segmentation_color_mode.ts, TypeScript, 115 lines
src/widget/shader_code_widget.ts, TypeScript, 253 lines
src/widget/shader_controls.ts, TypeScript, 380 lines
src/widget/star_button.ts, TypeScript, 27 lines
src/widget/tab_view.ts, TypeScript, 433 lines
src/widget/text_icon_button.ts, TypeScript, 29 lines
src/widget/text_input.ts, TypeScript, 70 lines
src/widget/tooltip.ts, TypeScript, 47 lines
src/widget/transfer_function.browser_test.ts, TypeScript, 411 lines
src/widget/transfer_function.ts, TypeScript, 2,005 lines
src/widget/vec3_entry_widget.ts, TypeScript, 96 lines
src/widget/virtual_list.ts, TypeScript, 516 lines
src/worker_rpc.ts, TypeScript, 446 lines
src/worker_rpc_context.ts, TypeScript, 21 lines
testdata/codec/crackle/generate_crackle_examples.py, Python, 94 lines
testdata/codec/npy/generate_npy_examples.py, Python, 69 lines
testdata/datasource/n5/generate_n5.py, Python, 69 lines
testdata/datasource/precomputed/generate_precomputed.py, Python, 46 lines
testdata/datasource/zarr/ome_zarr/generate_ome_zarr.py, Python, 85 lines
testdata/datasource/zarr/zarr_v2/from_zarr-python/generate-v2.py, Python, 199 lines
testdata/datasource/zarr/zarr_v3/examples/generate_zarr_v3.py, Python, 60 lines
testdata/datasource/zarr/zarr_v3/from_zarrita/generate-v3.py, Python, 553 lines
testdata/kvstore/icechunk/dump_icechunk.py, Python, 47 lines
testdata/kvstore/icechunk/generate_icechunk.py, Python, 132 lines
testdata/kvstore/ocdbt/generate_ocdbt.py, Python, 77 lines
testdata/kvstore/zip/generate_zip.py, Python, 22 lines
testdata/kvstore/zip/generate_zip64.py, Python, 94 lines
tests/codec/crackle.spec.ts, TypeScript, 196 lines
tests/datasource/completion.spec.ts, TypeScript, 138 lines
tests/datasource/deepzoom.browser_test.ts, TypeScript, 58 lines
tests/datasource/metadata_snapshot_test_util.ts, TypeScript, 41 lines
tests/datasource/n5.browser_test.ts, TypeScript, 24 lines
tests/datasource/nifti.browser_test.ts, TypeScript, 26 lines
tests/datasource/precomputed.browser_test.ts, TypeScript, 21 lines
tests/datasource/suggest_name.spec.ts, TypeScript, 33 lines
tests/datasource/test_util.ts, TypeScript, 184 lines
tests/datasource/zarr.browser_test.ts, TypeScript, 37 lines
tests/example_project_test/build_client.ts, TypeScript, 31 lines
tests/example_project_test/build_package.ts, TypeScript, 29 lines
tests/example_project_test/capture_build_screenshot.ts, TypeScript, 37 lines
tests/example_project_test/capture_dev_server_screenshot.ts, TypeScript, 83 lines
tests/example_project_test/capture_screenshot.ts, TypeScript, 60 lines
tests/example_project_test/client_test_options.ts, TypeScript, 27 lines
tests/example_project_test/compare_screenshots.ts, TypeScript, 32 lines
tests/example_project_test/config.ts, TypeScript, 141 lines
tests/example_project_test/install_node_modules.ts, TypeScript, 31 lines
tests/example_project_test/reset_screenshot_server.ts, TypeScript, 22 lines
tests/example_project_test/screenshot_comparison_server.ts, TypeScript, 85 lines
tests/fixtures/clear_cookies.ts, TypeScript, 31 lines
tests/fixtures/datasource_provider.ts, TypeScript, 35 lines
tests/fixtures/fake_gcs_server.ts, TypeScript, 120 lines
tests/fixtures/fake_s3_server.ts, TypeScript, 152 lines
tests/fixtures/fixture.ts, TypeScript, 54 lines
tests/fixtures/gl_browser.ts, TypeScript, 26 lines
tests/fixtures/gl_node.ts, TypeScript, 27 lines
tests/fixtures/http_server.ts, TypeScript, 50 lines
tests/fixtures/msw_browser.ts, TypeScript, 21 lines
tests/fixtures/msw_node.ts, TypeScript, 18 lines
tests/fixtures/shared_kvstore_context.ts, TypeScript, 50 lines
tests/fixtures/status_message_observer.ts, TypeScript, 89 lines
tests/fixtures/temp_directory.ts, TypeScript, 34 lines
tests/kvstore/gcs.spec.ts, TypeScript, 47 lines
tests/kvstore/gzip.spec.ts, TypeScript, 36 lines
tests/kvstore/http.spec.ts, TypeScript, 47 lines
tests/kvstore/icechunk.spec.ts, TypeScript, 292 lines
tests/kvstore/ngauth.browser_test.ts, TypeScript, 142 lines
tests/kvstore/ocdbt.spec.ts, TypeScript, 1,772 lines
tests/kvstore/s3.browser_test.ts, TypeScript, 67 lines
tests/kvstore/s3.spec.ts, TypeScript, 110 lines
tests/kvstore/test_data.ts, TypeScript, 56 lines
tests/kvstore/test_util.ts, TypeScript, 227 lines
tests/kvstore/url.spec.ts, TypeScript, 205 lines
tests/kvstore/zip.spec.ts, TypeScript, 182 lines
tests/util/clear_cookies.ts, TypeScript, 27 lines
tests/util/msw_request_log.ts, TypeScript, 128 lines
typings/index.d.ts, TypeScript, 2 lines
typings/nifti-reader-js.d.ts, TypeScript, 341 lines
typings/raw.d.ts, TypeScript, 4 lines
vitest.workspace.ts, TypeScript, 130 lines
LICENSE, License, 202 lines
README.md, Text, 177 lines

At the source: github.com/google/neuroglancer

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

4 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
1,301 scripts, each with its path and the digest of its content;
8 matches between paragraphs of the paper and lines of the code (method lexical-v1);
neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability statement

The original contributions presented in this study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 28 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 8 authors, 6 keywords, 1 funder, 193 references.

Cite

This paper

Wang, W., Yang, Y., Liu, Q., Gao, Y., Lv, Q., Zhang, K., Ning, J., & Sun, Y. (2026). Convergence-divergence circuits for multimodal integration of innate and learned opponent valences. Frontiers in systems neuroscience, 20, 1822122. https://doi.org/10.3389/fnsys.2026.1822122

BibTeX

@article{wang2026convergence, author = {Wang, Wenjing and Yang, Yaokai and Liu, Qiong and Gao, Yunming and Lv, Qiantao and Zhang, Kaiqi and Ning, Jing and Sun, Yi}, title = {{Convergence-divergence circuits for multimodal integration of innate and learned opponent valences}}, journal = {Frontiers in systems neuroscience}, year = {2026}, month = may, volume = {20}, pages = {1822122}, publisher = {Frontiers Media SA}, issn = {1662-5137}, doi = {10.3389/fnsys.2026.1822122}, url = {https://doi.org/10.3389/fnsys.2026.1822122}, pmid = {42181586}, pmcid = {PMC13190573} }

RIS

TY - JOUR AU - Wang, Wenjing AU - Yang, Yaokai AU - Liu, Qiong AU - Gao, Yunming AU - Lv, Qiantao AU - Zhang, Kaiqi AU - Ning, Jing AU - Sun, Yi TI - Convergence-divergence circuits for multimodal integration of innate and learned opponent valences T2 - Frontiers in systems neuroscience J2 - Front Syst Neurosci PY - 2026 DA - 2026/05/07 VL - 20 SP - 1822122 SN - 1662-5137 PB - Frontiers Media SA DO - 10.3389/fnsys.2026.1822122 UR - https://doi.org/10.3389/fnsys.2026.1822122 LA - en ER -

CSL-JSON

{ "id": "10.3389/fnsys.2026.1822122", "type": "article-journal", "title": "Convergence-divergence circuits for multimodal integration of innate and learned opponent valences", "container-title": "Frontiers in systems neuroscience", "author": [ { "family": "Wang", "given": "Wenjing" }, { "family": "Yang", "given": "Yaokai" }, { "family": "Liu", "given": "Qiong" }, { "family": "Gao", "given": "Yunming" }, { "family": "Lv", "given": "Qiantao" }, { "family": "Zhang", "given": "Kaiqi" }, { "family": "Ning", "given": "Jing" }, { "family": "Sun", "given": "Yi" } ], "container-title-short": "Front Syst Neurosci", "volume": "20", "page": "1822122", "DOI": "10.3389/fnsys.2026.1822122", "PMID": "42181586", "PMCID": "PMC13190573", "ISSN": "1662-5137", "publisher": "Frontiers Media SA", "URL": "https://doi.org/10.3389/fnsys.2026.1822122", "language": "en", "issued": { "date-parts": [ [ 2026, 5, 7 ] ] } }

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown

<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122.svg)](https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122)
<!-- oscr-badge:end -->

reStructuredText

.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122.svg
   :target: https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122
   :alt: OSCR

.. oscr-badge:end

HTML

<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122"><img src="https://openscicode.org/badge/doi/10.3389/fnsys.2026.1822122.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.

Convergence-divergence circuits for multimodal integration of innate and learned opponent valences.

Code ↔ Paper

Paper

The authors' code

Overview

Abstract

Repositories

lmcinnes/umap

navis-org/navis

aplbrain/dotmotif

google/neuroglancer

Tracing map

Data

Data availability statement

Versions

Version 1, 28 September 2026: the first record

Cite

This paper

Similar papers

Contribute

Claim this paper

Correct its record

Validate its tracing map

Add the badge to the code's README

Request its removal

Discussion, reproductions, activity