Hierarchical learning creates invariant schema within plastic neural networks.
The 6 matches
- [1] § Methods › Trial firing ↔ scripts/PlotMaker.py, lines 231–303 · score 0.69 · background firing, peak firing, activated firings, steepness, boundary, neurons
- [2] § Methods › Top models › Backpropagation networks ↔ scripts/PlotMaker.py, lines 545–608 · score 0.65 · MLPClassifier, max_iter, tanh activation, hidden layers, model, neuron
- [3] § Results › Distinct populations of hidden neurons segregate into sequential layers in hierarchical models ↔ scripts/PlotMaker.py, lines 545–608 · score 0.60 · peak firing, activated firing, event cells, hidden layer, NB, HB
- [4] § Results › Hierarchical networks converge to a stable schema robust to further training ↔ scripts/PlotMaker.py, lines 1–34 · score 0.60 · Concept space, Network robustness, event cell, Boundary cell, firing, ENN
- [5] § Results › Hierarchical schema is distillable into a concise interpretable circuit ↔ scripts/PlotMaker.py, lines 231–303 · score 0.56 · background firing, peak firing, Boundary cell, steepness, Event, activated
- [6] § Methods › Top models › Essence neural networks ↔ enn/network.py, lines 32–152 · score 0.55 · subconcept neurons, activation function, artificial, biases, tanh, class
Paper
Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
Python · 723 lines · 26 KB · no license · 5 matches
- '''
- Path: bdENS/repo/scripts/PlotMaker.py
- Boundary-Detecting ENN Project
- Results Plots
- - Figure 2a: Boundary Cell Firing
- - Figure 2b: Event Cell Firing
- - Figure 2c: Average Firing Rates
- - Figure 2d: Trial Firings
- - Figure 2e: Neuron Average Firing
- - Figure 3b: MLP Plots
- - Figure 3c: ENN Plots
- - Figure 3e: Network Robustness Plots
- - Figure 3f: Concept Space Plots
- Author: James R. Elder
- Institution: UTSW
- DOO: 11-25-2024
- LU: 11-25-2024
- Reference(s):
- - bdENS/release/FigureScripts/Fig2abc.py
- - bdENS/release/FigureScripts/Fig2d.py
- - bdENS/release/FigureScripts/Fig2e.py
- - bdENS/release/FigureScripts/Fig3b.py
- - bdENS/release/FigureScripts/Fig3c.py
- - bdENS/release/FigureScripts/Fig3e.py
- - bdENS/release/FigureScripts/Fig3f.py
- - Python 3.12.2
- - bdENSenv
- - 3.12.x-anaconda
- - linux-gnu (BioHPC)
- '''
- # Project variables
- user = "s181641"
- projID = "bdENS"
- labID = "Lin_lab"
- labAffiliation = "greencenter"
- projDir = "/work/{}/{}/{}/".format(labAffiliation, user, projID)
- # Subproject variables
- subprojID = "NetworkVisualization"
- branch = "repo"
- # System imports
- import time
- today = time.strftime("%m-%d-%Y")
- import os
- if os.getcwd() != projDir + branch:
- print("Changing to project directory")
- os.chdir(projDir + branch) # Change to the project directory
- import sys
- sys.path.append(projDir + branch) # Add the project directory to the system path
- # 3rd party imports
- import numpy as np
- import matplotlib.pyplot as plt
- # ENN imports
- # Local imports
- from utils import LumberJack as LJ
- from utils import NetworkVisualizer as NV
- from utils import commonFunctions as cF
- from utils import commonClasses as cC
- # Logging
- logMan = LJ.LoggingManager(subprojID)
- cF.updatePlotrcs(font="serif")
- # Variables
- globers = {
- "randomSeed": 42,
- "nTrials": 135,
- "nBins": 150,
- "msPerBin": 10,
- "boundaryBin": 50,
- }
- eventers = {
- "nNeurons": 36,
- "cellType": "HB",
- "peakFiring": 30.1,
- "peakStd": 5.5,
- }
- trialers = {
- "nNBTrials": 30,
- "nSBTrials": 75,
- "nHBTrials": 30
- }
- boundaryers = {
- "nNeurons": 42,
- "cellType": "B",
- "peakFiring": 19.7,
- "peakStd": 4.9
- }
- boundaryRecording = cF.loadNeuron(7, boundaryers["cellType"])
- eventRecording = cF.loadNeuron(2, eventers["cellType"])
- eventCellMeanRecordings = np.zeros((eventers["nNeurons"], globers["nBins"]))
- boundaryCellMeanRecordings = np.zeros((boundaryers["nNeurons"], globers["nBins"]))
- for i in range(1, eventers["nNeurons"]+1):
- neuronRecording = cF.loadNeuron(i, 'HB')
- neuronRecording = np.mean(neuronRecording[105:], axis=0)
- eventCellMeanRecordings[i-1] = neuronRecording
- eventCellsMeanRecordings = np.mean(eventCellMeanRecordings, axis=0)
- for i in range(1, boundaryers["nNeurons"]+1):
- neuronRecording = cF.loadNeuron(i, 'B')
- neuronRecording = np.mean(neuronRecording[30:], axis=0)
- boundaryCellMeanRecordings[i-1] = neuronRecording
- boundaryCellsMeanRecordings = np.mean(boundaryCellMeanRecordings, axis=0)
- figWidth = 8
- figHeight = 2.5
- fig, ax = plt.subplots(1,3, figsize=(figWidth, figHeight))
- randomSBTrials = np.random.choice(np.arange(trialers["nNBTrials"], trialers["nNBTrials"] + trialers["nSBTrials"]), 30, replace=False)
- boundaryRecordingSubSample = np.append(boundaryRecording[:30], boundaryRecording[randomSBTrials])
- boundaryRecordingSubSample = np.append(boundaryRecordingSubSample, boundaryRecording[105:])
- boundaryRecordingSubSample = np.reshape(boundaryRecordingSubSample, (-1, 150))
- ax[0].imshow(boundaryRecordingSubSample, aspect='auto', cmap="gray_r")
- ax[0].set_title("Boundary Cell (B)")
- randomSBTrials = np.random.choice(np.arange(trialers["nNBTrials"], trialers["nNBTrials"] + trialers["nSBTrials"]), 30, replace=False)
- eventRecordingSubSample = np.append(eventRecording[:30], eventRecording[randomSBTrials])
- eventRecordingSubSample = np.append(eventRecordingSubSample, eventRecording[105:])
- eventRecordingSubSample = np.reshape(eventRecordingSubSample, (-1, 150))
- ax[1].imshow(eventRecordingSubSample, aspect='auto', cmap="gray_r")
- ax[1].set_title("Event Cell (E)")
- ax[2].plot(boundaryCellsMeanRecordings, color=cC.cellColors["Boundary"], label="B", linestyle="-", linewidth=2.5)
- ax[2].plot(eventCellsMeanRecordings, color=cC.cellColors["Event"], label="E", linestyle="-", linewidth=2.5)
- ax[2].legend(loc="upper right", ncol=1, bbox_to_anchor=(1.1, 1.05))
- # Add early at the top of the plot on the left, and late at the top of the plot on the right
- ax[2].annotate("Early", xy=(globers["boundaryBin"], 0.27), xytext=(globers["boundaryBin"]+10, 0.27), textcoords="data", ha="center", va="center", fontsize=8, rotation=90)
- ax[2].annotate("Late", xy=(globers["boundaryBin"]+22, 0.275), xytext=(globers["boundaryBin"]+32, 0.275), textcoords="data", ha="center", va="center", fontsize=8, rotation=90)
- ax[2].set_title("Mean Firing Rate")
- for i in range(3):
- ax[i].spines['top'].set_visible(False)
- ax[i].spines['right'].set_visible(False)
- ax[i].set_xticks([0, 50, 100, 150], ["-0.5", "0.0", "0.5", "1.0"])
- ax[i].set_xlim(0, globers["nBins"])
- ax[i].set_xlabel("Time, relative to\nboundary (s)")
- ax[i].axvline(globers["boundaryBin"], color="black", linestyle="-")
- if i != 2:
- ax[i].set_ylabel("Trial")
- ax[i].set_ylim(90, 0)
- ax[i].set_yticks([0, 30, 60, 90], [])
- ax[i].axhspan(0, trialers["nNBTrials"], color=cC.conceptColors[0], alpha=0.1)
- ax[i].axhspan(trialers["nNBTrials"], trialers["nNBTrials"] + 30, color=cC.conceptColors[1], alpha=0.1)
- ax[i].axhspan(60, 90, color=cC.conceptColors[2], alpha=0.1)
- else:
- ax[i].set_yticks([0, 0.1, 0.2, 0.3], ["0", "1", "2", "3"])
- ax[i].set_ylabel("Firing Rate (Hz)")
- ax[i].axvline(globers["boundaryBin"]+22, color="black", linestyle="--")
- fig.tight_layout()
- fig.savefig(logMan.mediaDir + "/Fig2abc.png",
- transparent=True,
- dpi=300)
- plt.close(fig)
- # Figure 2d: Trial Firings
- # ENN imports
- from enn.network import Network
- import enn.learnBoundaries as lB
- # 3rd party imports
- from sklearn.neural_network import MLPClassifier
- from sklearn.model_selection import train_test_split
- """
- # Variables
- globers = {
- "randomSeed": 42,
- "nTrials": 135,
- "nBins": 150,
- "msPerBin": 10,
- "boundaryBin": 50,
- "trainTestSplit": 0.5,
- "nHiddenNeurons": 3,
- "nHiddenLayers": 2
- }
- mlpers = {
- "maxIter": 10000,
- "activation": "tanh",
- }
- eventers = {
- "nNeurons": 36,
- "cellType": "HB",
- "peakFiring": 30.1,
- "peakStd": 5.5,
- }
- trialers = {
- "nNBTrials": 30,
- "nSBTrials": 75,
- "nHBTrials": 30
- }
- enners = {
- "subconceptSelector": 0,
- }
- boundaryers = {
- "nNeurons": 42,
- "cellType": "B",
- "peakFiring": 19.7,
- "peakStd": 4.9
- }
- plotters = {
- "aTrial": 0.1,
- "cBoundary": "#E8BDCB",
- "cEvent": "#E5C8EE"
- }
- # Functions
- def activateFiring(_avgPeakFiring, _backgroundFiring):
- _scaleFactor = 0.5
- _xOffset = 1.5
- _steepness = 0.1 / np.std(np.mean(_backgroundFiring, axis=1))
- _yOffset = 0.5
- _normalizedFiring = _avgPeakFiring / np.mean(_backgroundFiring)
- return _scaleFactor * np.tanh(_steepness * (_normalizedFiring - _xOffset)) + _yOffset
- eventCellActivatedFirings = np.zeros((globers["nTrials"], eventers["nNeurons"]))
- lateBoundaryInds = []
- earlyEventInds = []
- cutoff = 22
- for i in range(eventers["nNeurons"]):
- recording = cF.loadNeuron(i+1, eventers["cellType"])
- preBoundaryFiring = recording[:,:globers["boundaryBin"]]
- peakFiring = recording[:,int(globers["boundaryBin"]+eventers["peakFiring"]-2*eventers["peakStd"]):int(globers["boundaryBin"] + eventers["peakFiring"]+2*eventers["peakStd"])]
- avgPeakFiring = np.mean(peakFiring, axis=1)
- if np.argmax(np.mean(recording[105:], axis=0)) <= globers["boundaryBin"] + cutoff:
- earlyEventInds.append(i)
- eventCellActivatedFirings[:,i] = activateFiring(avgPeakFiring, preBoundaryFiring)
- eventScorer = [-1, -1, 1]
- eventAvgClassFiring = [np.mean(eventCellActivatedFirings[:30], axis=0),
- np.mean(eventCellActivatedFirings[30:105], axis=0),
- np.mean(eventCellActivatedFirings[105:], axis=0)]
- eventCellEventness = np.dot(eventScorer, eventAvgClassFiring)
- eventnessInds = np.argsort(-eventCellEventness)
- eventCellActivatedFirings = eventCellActivatedFirings[:,eventnessInds]
- boundaryCellActivatedFirings = np.zeros((globers["nTrials"], boundaryers["nNeurons"]))
- for i in range(boundaryers["nNeurons"]):
- recording = cF.loadNeuron(i+1, boundaryers["cellType"])
- preBoundaryFiring = recording[:,:globers["boundaryBin"]]
- peakFiring = recording[:,int(globers["boundaryBin"]+boundaryers["peakFiring"]-2*boundaryers["peakStd"]):int(globers["boundaryBin"] + boundaryers["peakFiring"]+2*boundaryers["peakStd"])]
- avgPeakFiring = np.mean(peakFiring, axis=1)
- if np.argmax(np.mean(recording[30:], axis=0)) > globers["boundaryBin"] + cutoff:
- lateBoundaryInds.append(i)
- boundaryCellActivatedFirings[:,i] = activateFiring(avgPeakFiring, preBoundaryFiring)
- boundaryScorer = [-1, 1, 1]
- boundaryAvgClassFiring = [np.mean(boundaryCellActivatedFirings[:30], axis=0),
- np.mean(boundaryCellActivatedFirings[30:105], axis=0),
- np.mean( boundaryCellActivatedFirings[105:], axis=0)]
- boundaryCellBoundariness = np.dot(boundaryScorer, boundaryAvgClassFiring)
- boundaryInds = np.argsort(-boundaryCellBoundariness)
- boundaryCellActivatedFirings = boundaryCellActivatedFirings[:,boundaryInds]
- earlyCellFirings = []
- lateCellFirings = []
- for i in range(boundaryCellActivatedFirings.shape[1]):
- if i in lateBoundaryInds:
- lateCellFirings.append(boundaryCellActivatedFirings[:,i])
- else:
- earlyCellFirings.append(boundaryCellActivatedFirings[:,i])
- for i in range(eventCellActivatedFirings.shape[1]):
- if i in earlyEventInds:
- earlyCellFirings.append(eventCellActivatedFirings[:,i])
- else:
- lateCellFirings.append(eventCellActivatedFirings[:,i])
- earlyCellFirings = np.array(earlyCellFirings).T
- lateCellFirings = np.array(lateCellFirings).T
- earlyCellSubFirings = []
- lateCellSubFirings = []
- # Randomly sample 30 of the SB trials
- for i in range(earlyCellFirings.shape[1]):
- sbTrials = np.random.choice(range(30, 105), 30, replace=False)
- earlyCellSubFirings.append(np.append(earlyCellFirings[:30,i], earlyCellFirings[sbTrials,i]))
- earlyCellSubFirings[i] = np.append(earlyCellSubFirings[i], earlyCellFirings[105:,i])
- for i in range(lateCellFirings.shape[1]):
- sbTrials = np.random.choice(range(30, 105), 30, replace=False)
- lateCellSubFirings.append(np.append(lateCellFirings[:30,i], lateCellFirings[sbTrials,i]))
- lateCellSubFirings[i] = np.append(lateCellSubFirings[i], lateCellFirings[105:,i])
- earlyCellSubFirings = np.array(earlyCellSubFirings).T
- lateCellSubFirings = np.array(lateCellSubFirings).T
- xAll, yAll = cF.loadEmbeddings()
- xAll_NB = xAll[yAll == 0]
- xAll_SB = xAll[yAll == 1]
- xAll_HB = xAll[yAll == 2]
- nTrials = 20
- enn_l1Outputs = np.zeros((nTrials, 90, 3))
- enn_l2Outputs = np.zeros((nTrials, 90, 3))
- mlp_l1Outputs = np.zeros((nTrials, 90, 3))
- mlp_l2Outputs = np.zeros((nTrials, 90, 3))
- mlpAccs = np.zeros(nTrials)
- convThresh = 0.5
- for i in range(nTrials):
- xTrain, xTest, yTrain, yTest = train_test_split(xAll, yAll,
- train_size=globers["trainTestSplit"],
- random_state=globers["randomSeed"]+i)
- xAll_NB_TrialsInds = np.random.choice(range(len(xAll_NB)), 30, replace=False)
- xAll_SB_TrialsInds = np.random.choice(range(len(xAll_SB)), 30, replace=False)
- xAll_HB_TrialsInds = np.random.choice(range(len(xAll_HB)), 30, replace=False)
- xAll_Trials = np.concatenate((xAll_NB[xAll_NB_TrialsInds],
- xAll_SB[xAll_SB_TrialsInds],
- xAll_HB[xAll_HB_TrialsInds]), axis=0)
- mdlENN = lB.main_0(xTrain, yTrain, enners["subconceptSelector"], globers["nHiddenNeurons"])
- mdlMLP = MLPClassifier(hidden_layer_sizes=(globers["nHiddenNeurons"],globers["nHiddenNeurons"]), max_iter=mlpers["maxIter"],
- random_state=globers["randomSeed"]+i, activation=mlpers["activation"])
- mdlMLP.fit(xTrain, yTrain)
- mlpAccs[i] = mdlMLP.score(xTest, yTest)
- enn_l1Output = mdlENN.layers[0].compute_output(xAll_Trials, activate=True) # Tanh activation
- enn_l2Output = mdlENN.layers[1].compute_output(enn_l1Output, activate=True) # Tanh activation
- # Flip the sign of the firings rates if the average firing rate on NB is positive
- for i2 in range(3):
- if np.mean(enn_l1Output[:30, i2]) > 0:
- enn_l1Output[:, i2] = -enn_l1Output[:, i2]
- if np.mean(enn_l2Output[:30, i2]) > 0:
- enn_l2Output[:, i2] = -enn_l2Output[:, i2]
- enn_l1Outputs[i] = enn_l1Output
- enn_l2Outputs[i] = enn_l2Output
- if mlpAccs[i] > convThresh:
- mlp_l1Output_unAct = mdlMLP.coefs_[0].T @ xAll_Trials.T + mdlMLP.intercepts_[0].reshape(-1, 1)
- mlp_l1Output = np.tanh(mlp_l1Output_unAct) # Tanh activation
- mlp_l2Output_unAct = mdlMLP.coefs_[1].T @ mlp_l1Output + mdlMLP.intercepts_[1].reshape(-1, 1)
- mlp_l2Output = np.tanh(mlp_l2Output_unAct) # Tanh activation
- mlp_l1Output = mlp_l1Output.T
- mlp_l2Output = mlp_l2Output.T
- for i2 in range(3):
- if np.mean(mlp_l1Output[:30, i2]) > 0:
- mlp_l1Output[:, i2] = -mlp_l1Output[:, i2]
- if np.mean(mlp_l2Output[:30, i2]) > 0:
- mlp_l2Output[:, i2] = -mlp_l2Output[:, i2]
- mlp_l1Outputs[i] = mlp_l1Output
- mlp_l2Outputs[i] = mlp_l2Output
- # Remove unconverged networks
- mlp_l1Outputs = mlp_l1Outputs[mlpAccs > convThresh]
- mlp_l2Outputs = mlp_l2Outputs[mlpAccs > convThresh]
- enn_l1Outputs = enn_l1Outputs[mlpAccs > convThresh]
- enn_l2Outputs = enn_l2Outputs[mlpAccs > convThresh]
- mlpL1Firings = mlp_l1Outputs.reshape(-1, 90, 3)
- mlpL2Firings = mlp_l2Outputs.reshape(-1, 90, 3)
- ennL1Firings = enn_l1Outputs.reshape(-1, 90, 3)
- ennL2Firings = enn_l2Outputs.reshape(-1, 90, 3)
- mlpL1Cells = []
- mlpL2Cells = []
- ennL1Cells = []
- ennL2Cells = []
- for i in range(ennL1Firings.shape[0]):
- for i2 in range(3):
- mlpL1Cells.append(mlpL1Firings[i,:,i2])
- mlpL2Cells.append(mlpL2Firings[i,:,i2])
- ennL1Cells.append(ennL1Firings[i,:,i2])
- ennL2Cells.append(ennL2Firings[i,:,i2])
- mlpL1Cells = np.array(mlpL1Cells)
- mlpL2Cells = np.array(mlpL2Cells)
- ennL1Cells = np.array(ennL1Cells)
- ennL2Cells = np.array(ennL2Cells)
- mlpL1Cells = np.reshape(mlpL1Cells, (-1, 90))
- mlpL2Cells = np.reshape(mlpL2Cells, (-1, 90))
- ennL1Cells = np.reshape(ennL1Cells, (-1, 90))
- ennL2Cells = np.reshape(ennL2Cells, (-1, 90))
- # Sort the cells by average firing rate [Update this for Milo]
- # Need to put Boundary like cells first, then Event like cells, then the rest
- def sortCells(_cells):
- _threshold = 1.5
- _boundaryScorer = [-1, 1, 1]
- _eventScorer = [-1, -1, 1]
- _boundaryCells = []
- _eventCells = []
- _otherCells = []
- for i in range(_cells.shape[0]):
- boundaryScore = np.dot(_boundaryScorer, [np.mean(_cells[i,:30]), np.mean(_cells[i,30:60]), np.mean(_cells[i,60:])])
- eventScore = np.dot(_eventScorer, [np.mean(_cells[i,:30]), np.mean(_cells[i,30:60]), np.mean(_cells[i,60:])])
- if boundaryScore > eventScore and boundaryScore > _threshold:
- _boundaryCells.append(_cells[i])
- elif eventScore > boundaryScore and eventScore > _threshold:
- _eventCells.append(_cells[i])
- else:
- _otherCells.append(_cells[i])
- _boundaryCells = np.array(_boundaryCells)
- _boundaryCells = np.reshape(_boundaryCells, (-1, 90))
- _eventCells = np.array(_eventCells)
- _eventCells = np.reshape(_eventCells, (-1, 90))
- _otherCells = np.array(_otherCells)
- _otherCells = np.reshape(_otherCells, (-1, 90))
- # Sort within the boundary and event cells
- _boundaryCells = _boundaryCells[np.argsort(-np.mean(_boundaryCells[:,30:], axis=1))]
- _eventCells = _eventCells[np.argsort(-np.mean(_eventCells[:,60:], axis=1))]
- _otherCells = _otherCells[np.argsort(-np.mean(_otherCells, axis=1))]
- print("Boundary Cells: {}".format(_boundaryCells.shape))
- print("Event Cells: {}".format(_eventCells.shape))
- print("Other Cells: {}".format(_otherCells.shape))
- # Append all cells
- _allCells = np.concatenate((_boundaryCells, _eventCells, _otherCells), axis=0)
- print("All Cells: {}".format(_allCells.shape))
- return _allCells
- ennL1Cells = sortCells(ennL1Cells)
- ennL2Cells = sortCells(ennL2Cells)
- mlpL1Cells = sortCells(mlpL1Cells)
- mlpL2Cells = sortCells(mlpL2Cells)
- figWidth = 7
- figHeight = 2.5
- fig, ax = plt.subplots(1, 6, figsize=(figWidth, figHeight), tight_layout=True, sharey=True)
- _aspect=1
- ax[0].imshow(earlyCellSubFirings, cmap="gray_r",
- aspect=_aspect,
- vmin=0, vmax=1, interpolation="none")
- ax[1].imshow(ennL1Cells.T, cmap="gray_r",
- aspect=_aspect,
- vmin=-1, vmax=1, interpolation="none")
- ax[2].imshow(mlpL1Cells.T, cmap="gray_r",
- aspect=_aspect, vmin=-1, vmax=1, interpolation="none")
- ax[3].imshow(lateCellSubFirings, cmap="gray_r",
- aspect=_aspect, vmin=0, vmax=1, interpolation="none")
- ax[4].imshow(ennL2Cells.T, cmap="gray_r",
- aspect=_aspect, vmin=-1, vmax=1, interpolation="none")
- ax[5].imshow(mlpL2Cells.T, cmap="gray_r",
- aspect=_aspect, vmin=-1, vmax=1, interpolation="none")
- titles = ["MTL", "ENN", "Backprop"]
- for i in range(6):
- ax[i].set_yticks([0,30,60,90], labels=[])
- ax[i].axhspan(0, 30, color=cC.conceptColors[0], alpha=0.1)
- ax[i].axhspan(30, 30 + 30, color=cC.conceptColors[1], alpha=0.1)
- ax[i].axhspan(30 + 30, 30 + 30 + 30, color=cC.conceptColors[2], alpha=0.1)
- ax[i].set_ylim(89, 0)
- ax[i].set_xticks([], labels=[])
- ax[i].set_title(titles[i%3])
- ax[1].set_xlabel("Early / Layer 1 Neurons")
- ax[4].set_xlabel("Late / Layer 2 Neurons")
- fig.tight_layout()
- fig.savefig(logMan.mediaDir + "/Fig2d.png",
- transparent=True,
- dpi=300)
- plt.close(fig)
- """
- # Figure 2e: Neuron Average Firing
- # Functions
- def activateFiring(_avgPeakFiring, _backgroundFiring):
- _scaleFactor = 0.5
- _xOffset = 1.5
- _steepness = 0.1 / np.std(np.mean(_backgroundFiring, axis=1))
- _yOffset = 0.5
- _normalizedFiring = _avgPeakFiring / np.mean(_backgroundFiring)
- return _scaleFactor * np.tanh(_steepness * (_normalizedFiring - _xOffset)) + _yOffset
- def processNeurons(_nNeurons, _cellType, _peakFiring, _peakStd, globers):
- _activatedFirings = np.zeros((globers["nTrials"], _nNeurons))
- for i in range(_nNeurons):
- _recording = cF.loadNeuron(i + 1, _cellType)
- _preBoundaryFiring = _recording[:, :globers["boundaryBin"]]
- _peakFiringRange = _recording[:, int(globers["boundaryBin"] + _peakFiring - 2*_peakStd):int(globers["boundaryBin"] + _peakFiring + 2*_peakStd)]
- _avgPeakFiring = np.mean(_peakFiringRange, axis=1)
- _activatedFirings[:, i] = activateFiring(_avgPeakFiring, _preBoundaryFiring)
- return _activatedFirings
- def computeStats(_activatedFirings):
- avgNB = np.mean(_activatedFirings[:30], axis=0)
- avgSB = np.mean(_activatedFirings[30:30 + 75], axis=0)
- avgHB = np.mean(_activatedFirings[30 + 75:], axis=0)
- stdNB = np.std(_activatedFirings[:30], axis=0)
- stdSB = np.std(_activatedFirings[30:30 + 75], axis=0)
- stdHB = np.std(_activatedFirings[30 + 75:], axis=0)
- return [avgNB, avgSB, avgHB], [stdNB, stdSB, stdHB]
- eventers = {
- "nNeurons": 36,
- "cellType": "HB",
- "peakFiring": 30.1,
- "peakStd": 5.5,
- }
- boundaryers = {
- "nNeurons": 42,
- "cellType": "B",
- "peakFiring": 19.7,
- "peakStd": 4.9
- }
- globers = {
- "randomSeed": 42,
- "nTrials": 135,
- "nBins": 150,
- "msPerBin": 10,
- "boundaryBin": 50,
- "nHiddenNeurons": 3,
- "trainTestSplit": 0.5,
- }
- eventCellActivatedFirings = processNeurons(eventers["nNeurons"], eventers["cellType"], eventers["peakFiring"], eventers["peakStd"], globers)
- boundaryCellActivatedFirings = processNeurons(boundaryers["nNeurons"], boundaryers["cellType"], boundaryers["peakFiring"], boundaryers["peakStd"], globers)
- eventCellActivatedFiringsAvg, eventCellActivatedFiringsStd = computeStats(eventCellActivatedFirings)
- boundaryCellActivatedFiringsAvg, boundaryCellActivatedFiringsStd = computeStats(boundaryCellActivatedFirings)
- xAll, yAll = cF.loadEmbeddings()
- xAll_NB = xAll[yAll == 0]
- xAll_SB = xAll[yAll == 1]
- xAll_HB = xAll[yAll == 2]
- nNetworks = 20
- enn_l1Outputs = np.zeros((nNetworks, 90, 3))
- enn_l2Outputs = np.zeros((nNetworks, 90, 3))
- mlp_l1Outputs = np.zeros((nNetworks, 90, 3))
- mlp_l2Outputs = np.zeros((nNetworks, 90, 3))
- mlpAccs = np.zeros(nNetworks)
- convThresh = 0.5
- for i in range(nNetworks):
- xTrain, xTest, yTrain, yTest = train_test_split(xAll, yAll,
- train_size=globers["trainTestSplit"],
- random_state=42+i)
- mdlMLP = MLPClassifier(hidden_layer_sizes=(3,3), max_iter=10000, random_state=42+i, activation="tanh")
- mdlMLP.fit(xTrain, yTrain)
- mlpAccs[i] = mdlMLP.score(xTest, yTest)
- mdlENN = lB.main_0(xTrain, yTrain, 0, globers["nHiddenNeurons"])
- xAll_NB_TrialsInds = np.random.choice(range(len(xAll_NB)), 30, replace=False)
- xAll_SB_TrialsInds = np.random.choice(range(len(xAll_SB)), 30, replace=False)
- xAll_HB_TrialsInds = np.random.choice(range(len(xAll_HB)), 30, replace=False)
- xAll_Trials = np.concatenate((xAll_NB[xAll_NB_TrialsInds],
- xAll_SB[xAll_SB_TrialsInds],
- xAll_HB[xAll_HB_TrialsInds]), axis=0)
- enn_l1Output = mdlENN.layers[0].compute_output(xAll_Trials, activate=True) # Tanh activation
- enn_l2Output = mdlENN.layers[1].compute_output(enn_l1Output, activate=True) # Tanh activation
- # Flip the sign of the firings rates if the average firing rate on NB is positive
- for i2 in range(3):
- if np.mean(enn_l1Output[:30, i2]) > 0:
- enn_l1Output[:, i2] = -enn_l1Output[:, i2]
- if np.mean(enn_l2Output[:30, i2]) > 0:
- enn_l2Output[:, i2] = -enn_l2Output[:, i2]
- enn_l1Outputs[i] = enn_l1Output
- enn_l2Outputs[i] = enn_l2Output
- if mlpAccs[i] > convThresh:
- mlp_l1Output_unAct = mdlMLP.coefs_[0].T @ xAll_Trials.T + mdlMLP.intercepts_[0].reshape(-1, 1)
- mlp_l1Output = np.tanh(mlp_l1Output_unAct) # Tanh activation
- mlp_l2Output_unAct = mdlMLP.coefs_[1].T @ mlp_l1Output + mdlMLP.intercepts_[1].reshape(-1, 1)
- mlp_l2Output = np.tanh(mlp_l2Output_unAct) # Tanh activation
- mlp_l1Output = mlp_l1Output.T
- mlp_l2Output = mlp_l2Output.T
- for i2 in range(3):
- if np.mean(mlp_l1Output[:30, i2]) > 0:
- mlp_l1Output[:, i2] = -mlp_l1Output[:, i2]
- if np.mean(mlp_l2Output[:30, i2]) > 0:
- mlp_l2Output[:, i2] = -mlp_l2Output[:, i2]
- mlp_l1Outputs[i] = mlp_l1Output
- mlp_l2Outputs[i] = mlp_l2Output
- # Remove unconverged networks
- mlp_l1Outputs = mlp_l1Outputs[mlpAccs > convThresh]
- mlp_l2Outputs = mlp_l2Outputs[mlpAccs > convThresh]
- enn_l1Outputs = enn_l1Outputs[mlpAccs > convThresh]
- enn_l2Outputs = enn_l2Outputs[mlpAccs > convThresh]
- enn_l1AvgNBFiring = np.mean(enn_l1Outputs[:,:30,:], axis=1)
- enn_l1AvgSBFiring = np.mean(enn_l1Outputs[:,30:30+30,:], axis=1)
- enn_l1AvgHBFiring = np.mean(enn_l1Outputs[:,30+30:,:], axis=1)
- enn_l2AvgNBFiring = np.mean(enn_l2Outputs[:,:30,:], axis=1)
- enn_l2AvgSBFiring = np.mean(enn_l2Outputs[:,30:30+30,:], axis=1)
- enn_l2AvgHBFiring = np.mean(enn_l2Outputs[:,30+30:,:], axis=1)
- ennl1StdNBFiring = np.std(enn_l1Outputs[:,:30,:], axis=1)
- ennl1StdSBFiring = np.std(enn_l1Outputs[:,30:30+30,:], axis=1)
- ennl1StdHBFiring = np.std(enn_l1Outputs[:,30+30:,:], axis=1)
- ennl2StdNBFiring = np.std(enn_l2Outputs[:,:30,:], axis=1)
- ennl2StdSBFiring = np.std(enn_l2Outputs[:,30:30+30,:], axis=1)
- ennl2StdHBFiring = np.std(enn_l2Outputs[:,30+30:,:], axis=1)
- mlp_l1AvgNBFiring = np.mean(mlp_l1Outputs[:,:30,:], axis=1)
- mlp_l1AvgSBFiring = np.mean(mlp_l1Outputs[:,30:30+30,:], axis=1)
- mlp_l1AvgHBFiring = np.mean(mlp_l1Outputs[:,30+30:,:], axis=1)
- mlp_l2AvgNBFiring = np.mean(mlp_l2Outputs[:,:30,:], axis=1)
- mlp_l2AvgSBFiring = np.mean(mlp_l2Outputs[:,30:60,:], axis=1)
- mlp_l2AvgHBFiring = np.mean(mlp_l2Outputs[:,30+30:,:], axis=1)
- mlp_l1StdNBFiring = np.std(mlp_l1Outputs[:,:30,:], axis=1)
- mlp_l1StdSBFiring = np.std(mlp_l1Outputs[:,30:60,:], axis=1)
- mlp_l1StdHBFiring = np.std(mlp_l1Outputs[:,30+30:,:], axis=1)
- mlp_l2StdNBFiring = np.std(mlp_l2Outputs[:,:30,:], axis=1)
- mlp_l2StdSBFiring = np.std(mlp_l2Outputs[:,30:60,:], axis=1)
- mlp_l2StdHBFiring = np.std(mlp_l2Outputs[:,30+30:,:], axis=1)
- # Plot the results
- figWidth = 8
- figHeight = 4
- fig, ax = plt.subplots(1, 3, figsize=(figWidth, figHeight), subplot_kw={'projection': '3d'})
- ax[0].scatter(eventCellActivatedFiringsAvg[0], eventCellActivatedFiringsAvg[1], eventCellActivatedFiringsAvg[2], color=cC.cellColors["Event"], label='Event Cell', alpha=1)
- ax[0].scatter(boundaryCellActivatedFiringsAvg[0], boundaryCellActivatedFiringsAvg[1], boundaryCellActivatedFiringsAvg[2], color=cC.cellColors["Boundary"], label='Boundary Cell', alpha=1)
- ax[0].legend(bbox_to_anchor=(0., 2), loc='upper left', ncols=2)
- ax[1].scatter(enn_l1AvgNBFiring, enn_l1AvgSBFiring, enn_l1AvgHBFiring, color='blue', label='Layer 1', alpha=1)
- ax[1].scatter(enn_l2AvgNBFiring, enn_l2AvgSBFiring, enn_l2AvgHBFiring, color='red', label='Layer 2', alpha=1)
- #ax[1].legend(bbox_to_anchor=(0.5, 2), loc='upper left', ncols=2)
- ax[2].scatter(mlp_l1AvgNBFiring, mlp_l1AvgSBFiring, mlp_l1AvgHBFiring, color='blue', label='Layer 1', alpha=1)
- ax[2].scatter(mlp_l2AvgNBFiring, mlp_l2AvgSBFiring, mlp_l2AvgHBFiring, color='red', label='Layer 2', alpha=1)
- ax[2].legend(bbox_to_anchor=(-0.5, 2), loc='upper left', ncols=2)
- ax[0].set_title("MTL")
- ax[1].set_title("ENN")
- ax[2].set_title("Backprop")
- ax[0].set_xlim([0, 1])
- ax[0].set_ylim([0, 1])
- ax[0].set_zlim([0, 1])
- ax[0].set_xticks([0, 0.5, 1])
- ax[0].set_yticks([0, 0.5, 1])
- ax[0].set_zticks([0, 0.5, 1])
- for i in range(3):
- ax[i].set_xlabel("{} Firing".format(r'$\overline{NB}$'))
- ax[i].set_ylabel("{} Firing".format(r'$\overline{SB}$'))
- ax[i].set_zlabel("{} Firing".format(r'$\overline{HB}$'))
- for i in range(1,3):
- ax[i].set_xlim([-1, 1])
- ax[i].set_ylim([-1, 1])
- ax[i].set_zlim([-1, 1])
- ax[i].set_xticks([-1, 0, 1])
- ax[i].set_yticks([-1, 0, 1])
- ax[i].set_zticks([-1, 0, 1])
- fig.subplots_adjust(wspace=0.75)
- fig.savefig(logMan.mediaDir + "/Fig2e.png",
- transparent=True,
- dpi=300)
- plt.close(fig)
PlotMaker.py at commit 8aee36c, no license · at the source
Overview
- Green Center for Systems Biology, University of Texas Southwestern Medical Center,Dallas, TX 75390 USA
- Lyda Hill Dept. of Bioinformatics, University of Texas Southwestern Medical Center,Dallas, TX 75390 USA
- Molecular Biophysics Program, University of Texas Southwestern Medical Center,Dallas, TX 75390 USA
- Department of Biomedical Engineering, University of California at Davis,Davis, CA 95616 USA
- Department of Neurological Surgery, UC Davis Health,Davis, CA 95616 USA
- Biomedical Engineering and Neuroscience, Harvard University,Cambridge, MA 02138 USA
- Department of Neurosurgery, Cedars-Sinai Medical Center,Los Angeles, CA 90048 USA
- Department of Neurology, Cedars-Sinai Medical Center,Los Angeles, CA 90048 USA
- Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center,Los Angeles, CA 90048 USA
- Division of Biology and Biological Engineering, California Institute of Technology,Pasadena, CA 91125 USA
- Department of Biophysics, University of Texas Southwestern Medical Ctr.,Dallas, TX 75390 USA
- Center for Alzheimer’s and Neurodegenerative Diseases, University of Texas Southwestern Medical Center,Dallas, TX 75390 USA
Abstract
The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.
Repository
Its files are read in the Code ↔ Paper reader above, with 6 matches between paragraphs and lines of code.
jre411/bdENN
8aee36c26295ac90ceec784a66da1925c225d34a, 4 February 2025Availability: 1 check, the latest on 27 September 2026: the link answers
- 27 September 2026: the link answers
25 files
- enn/
__init__.py , Python, 1 line - enn/
enn_svm.py , Python, 431 lines - enn/
layer.py , Python, 587 lines - enn/
learnBoundaries.py , Python, 111 lines - enn/
network.py , Python, 496 lines, 1 match - enn/
subclass.py , Python, 19 lines - enn/
train_enn.py , Python, 2,333 lines - scripts/
ConceptSpaceRetention.py , Python, 295 lines - scripts/
NetworkVisualization.py , Python, 262 lines - scripts/
NoiseInjection.py , Python, 174 lines - scripts/
ParameterRobustness.py , Python, 217 lines - scripts/
PlotMaker.py , Python, 723 lines, 5 matches - scripts/
VariableSplitsAndReprodu , Python, 273 linescibility.py - scripts/
VariableTrainingData.py , Python, 115 lines - scripts/
bdENC.py , Python, 181 lines - scripts/
sFig1.py , Python, 100 lines - scripts/
sFig2.py , Python, 113 lines - scripts/
sFig3.py , Python, 104 lines - scripts/
sFig4.py , Python, 171 lines - utils/
LumberJack.py , Python, 81 lines - utils/
NetworkVisualizer.py , Python, 298 lines - utils/
__init__.py , Python, 1 line - utils/
commonClasses.py , Python, 16 lines - utils/
commonFunctions.py , Python, 131 lines - README.md, Text, 46 lines
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 24 scripts, each with its path and the digest of its content;
- 6 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
No dataset and no data link were found in the paper.
Code and data availability statement
The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:
- it points to the authors' code: jre411/
bdENN
Read it in the paper: doi.org/10.1007/s10827-026-00940-x.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 2, 28 September 2026
- Publisher: n/a → Springer Science+Business Media
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 5 authors, 2 keywords, 10 MeSH terms, 1 funder, 37 references.
Cite
This paper
Elder, J. R., Zheng, J., Shimelis, L. B., Rutishauser, U., & Lin, M. M. (2026). Hierarchical learning creates invariant schema within plastic neural networks. Journal of computational neuroscience, 54(3), 503-514. https://
BibTeX
@article{elder2026hierar
author = {Elder, James R. and Zheng, Jie and Shimelis, Lydia B. and Rutishauser, Ueli and Lin, Milo M.},
title = {{Hierarchical learning creates invariant schema within plastic neural networks}},
journal = {Journal of computational neuroscience},
year = {2026},
month = jun,
volume = {54},
number = {3},
pages = {503--514},
publisher = {Springer Science+Business Media},
issn = {0929-5313},
doi = {10.1007/
url = {https://
pmid = {42377689},
pmcid = {PMC13588848}
}
RIS
TY - JOUR
AU - Elder, James R.
AU - Zheng, Jie
AU - Shimelis, Lydia B.
AU - Rutishauser, Ueli
AU - Lin, Milo M.
TI - Hierarchical learning creates invariant schema within plastic neural networks
T2 - Journal of computational neuroscience
J2 - J Comput Neurosci
PY - 2026
DA - 2026/
VL - 54
IS - 3
SP - 503
EP - 514
SN - 0929-5313
PB - Springer Science+Business Media
DO - 10.1007/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1007/
"type": "article-journal",
"title": "Hierarchical learning creates invariant schema within plastic neural networks",
"container-title": "Journal of computational neuroscience",
"author": [
{
"family": "Elder",
"given": "James R."
},
{
"family": "Zheng",
"given": "Jie"
},
{
"family": "Shimelis",
"given": "Lydia B."
},
{
"family": "Rutishauser",
"given": "Ueli"
},
{
"family": "Lin",
"given": "Milo M."
}
],
"container-title-short":
"volume": "54",
"issue": "3",
"page": "503-514",
"DOI": "10.1007/
"PMID": "42377689",
"PMCID": "PMC13588848",
"ISSN": "0929-5313",
"publisher": "Springer Science+Business Media",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
30
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1038/s42003-026-10169-0 [code]
- Shared representations in brains and models reveal a two-route cortical organization during scene perception.Journal: Communications biologyIn common: scikit-learn, pandas, SciPy, 2 other tools, 3 references
- [2] doi:10.1162/imag.a.1207 [code]
- Investigating the temporal dynamics and modeling of mid-level feature representations in humans.Journal: Imaging neuroscience (Cambridge, Mass.)In common: scikit-learn, pandas, SciPy, 2 other tools, 3 references
- [3] doi:10.1186/s13321-026-01177-7 [code]
- A pipeline for developing AI-driven models to predict molecular initiating events: a case study on neural tube defects.Journal: Journal of cheminformaticsIn common: scikit-learn, pandas, SciPy, 2 other tools, 2 references
- [4] doi:10.1371/journal.pcbi.1014656 [code]
- Contrastive learning to fine-tune feature extraction models for the visual cortex.Journal: PLoS computational biologyIn common: scikit-learn, pandas, SciPy, 2 other tools, 2 references
- [5] doi:10.1038/s42003-026-10094-2 [code]
- E/
I imbalance and internal noise cause weak neural representations and face recognition challenges in ASD. Journal: Communications biologyIn common: scikit-learn, pandas, SciPy, 2 other tools, 2 references - [6] doi:10.1016/j.isci.2026.116439 [code]
- Decoding the role of transcriptomic clocks in the human prefrontal cortex.Journal: iScienceIn common: Keras, scikit-learn, pandas, 3 other tools, 1 reference
- [7] doi:10.1038/s41597-026-07248-6 [code]
- A large-scale fMRI dataset for vision-language semantic association.Journal: Scientific dataIn common: scikit-learn, pandas, SciPy, 2 other tools, 2 references
- [8] doi:10.1016/j.bpj.2026.06.005 [code]
- Curvature-based machine-learning method for automated segmentation of dendritic spines.Journal: Biophysical journalIn common: Keras, scikit-learn, pandas, 3 other tools, 1 reference
- [9] doi:10.1038/s41467-026-72687-z [code]
- Chordless cycle filtrations for dimensionality detection in complex networks via topological data analysis.Journal: Nature communicationsIn common: Keras, scikit-learn, pandas, 3 other tools, 1 reference
- [10] doi:10.1038/s41598-026-52672-8 [code]
- Validation of portable, semi-dry electrode-based electroencephalography device for its application in brain-computer interface solutions.Journal: Scientific reportsIn common: Keras, scikit-learn, SciPy, 2 other tools, 1 reference
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 1 repository of the authors' code, each at its verified commit and with its license, 24 scripts, and 6 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:8d59f11f4ae0a0af…
Add the badge to its README
The badge links the code to this page. Copy one of these into the README of the paper's code: only you decide where it goes, and nothing is changed for you.
Markdown
[, paste the snippet at the top, then “Commit changes…” and, to review it first, “Create a new branch and start a pull request”. You open the pull request; OSCR asks for no permission.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.
Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
