DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice.
5 blocks of code here have a permanent identifier (oscr:b:…), listed with the matches below: citable for good, with a page of its own. Select a match and press Cite to copy it. What a block identifier is.
The 5 matches
- [1] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_DeepFace_6s_6f.m, lines 509 to 580 · score 0.68 · auto regressive, processed brain, brain features, vector, linearly, filter · block
oscr:b:WB3X7ZMJ8640T-A· Report a mismatch - [2] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_8m_100Hz.m, lines 269 to 314 · score 0.63 · velocity features, raw feature, window aggregation, frame, Facemap, prediction · block
oscr:b:DWJVEYH0XS3BB-Q· Report a mismatch - [3] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 379 to 421 · score 0.62 · velocity features, raw feature, window aggregation, frame, Facemap, prediction · block
oscr:b:M00XPP0K2YQ2V-6· Report a mismatch - [4] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 634 to 721 · score 0.60 · eye features, mouth features, centroid, nose, dynamic, prediction · block
oscr:b:NZX3BZPCPK7QY-E· Report a mismatch - [5] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 1 to 44 · score 0.57 · corresponding brain, face brain, hemodynamic, FaceMap, Facial, tracking · block
oscr:b:JJYPQVDBBAT4X-A· Report a mismatch
Paper
Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
MATLAB · 736 lines · 29 KB · CC-BY-4.0 · 3 matches
- function FaceBrainPredictionGUI_Facemap_NoEnsemble_winsorizedR2_ko
- % FaceBrainPredictionGUI_Facemap_NoEnsemble_winsorizedR2.m
- %
- % Overview:
- %
- % The GUI includes the following functionalities:
- % 1. Set Data Label: Specify a label for the dataset.
- % 2. Change Save Folder: Select a folder to save outputs, models, graphs, and test data.
- % 3. Add Training Pairs: Select face CSV and corresponding brain MAT files for training.
- % 4. Clear Training Pairs: Remove current training data from memory.
- % 5. Add Testing Pairs: Select face CSV and corresponding brain MAT files for testing.
- % 6. Clear Testing Pairs: Remove current testing data from memory.
- % 7. Clear In-Memory Model: Reset the current model.
- % 8. Load Pre-Trained Model: Load a previously saved model (Neural Network or Ensemble).
- % 9. Train New Model: Train a new Neural Network model using the training pairs.
- % 10. Test Model: Run the model on testing pairs. For each test pair:
- % - Computes performance metrics: RMSE, standard R², Winsorized R² (Filtered R²), and MAE.
- % - Generates a figure comparing actual and predicted signals.
- % - Saves the figure as both .fig and .png files.
- % - Saves test data and metrics in a .mat file.
- % - Writes overall test metrics to an Excel file.
- % 11. Save Pairs to Disk: Save current training/testing pairs to a .mat backup file.
- % 12. Load Pairs from Disk: Load training/testing pairs from a backup file.
- % 13. Exit: Exit the application.
- %
- % Instructions for New Users:
- % - Prepare your face data as CSV files that include columns for tracking facial features.
- % - Prepare your brain data as MAT files containing a variable with hemodynamic signals.
- % The script automatically detects the relevant field (e.g., 'hemodynamic').
- % - Follow the menu options to add training/testing pairs.
- % - Train your model and then test it; the script will generate and save graphs and metrics.
- %
- % Outputs:
- % - Trained model saved as a .mat file.
- % - Test graphs saved as .fig and .png files.
- % - Test data and metrics saved as individual .mat files.
- % - An Excel file summarizing test metrics.
- %
- % Note:
- % - The Winsorized R² (Filtered R²) calculation ignores the top 5% of errors to
- % reduce the impact of outliers.
- clc; clear;
- %% Default Parameters
- dataLabel = 'withaddbrain';
- savePath = pwd; % Default: current folder
- fsBrain = 20; % Brain sampling rate (Hz)
- fsFace = 120; % Face sampling rate (Hz)
- bandpassLowCut = 0.1; % For brain band-pass filtering
- bandpassHighCut = 9.99;
- butterOrder = 2;
- movAvgBrainSec = 1; % 1 s moving average for brain smoothing
- ignoreInitialSec = 5; % Ignore first 5 s of both signals
- faceWindowSize = 5; % Number of face frames per time-window aggregator
- nnHiddenNeurons = 5;
- nnEpochs = 100;
- % ensLearningCycles = 50; % Not used
- % Filenames for saving test metrics and pairs backup
- testMetricsFile = 'TestMetrics.xlsx';
- pairsBackupFile = [dataLabel '_pairsBackup.mat'];
- %% Data in Memory
- trainingPairs = [];
- testingPairs = [];
- %% Model
- currentModel = [];
- currentModelType = ''; % 'Neural Network' or 'Ensemble'
- %% Main Menu Loop
- while true
- choice = menu('Select an option', ...
- '1. Set Data Label', ...
- '2. Change Save Folder', ...
- '3. Add Training Pairs (Face CSV + Brain MAT)', ...
- '4. Clear Training Pairs', ...
- '5. Add Testing Pairs (Face CSV + Brain MAT)', ...
- '6. Clear Testing Pairs', ...
- '7. Clear In-Memory Model', ...
- '8. Load Pre-Trained Model', ...
- '9. Train New Model', ...
- '10. Test Model', ...
- '11. Save Pairs to Disk', ...
- '12. Load Pairs from Disk', ...
- '13. Exit');
- switch choice
- case 1
- prompt = {'Enter data label (e.g. withaddbrain):'};
- dlg_title = 'Set Data Label';
- answer = inputdlg(prompt, dlg_title, 1, {dataLabel});
- if ~isempty(answer)
- dataLabel = answer{1};
- fprintf('Data label set to: %s\n', dataLabel);
- end
- case 2
- newFolder = uigetdir([], 'Select folder for saving outputs/models');
- if isequal(newFolder, 0)
- disp('No folder selected. Save folder unchanged.');
- else
- savePath = newFolder;
- fprintf('Save folder changed to: %s\n', savePath);
- end
- case 3
- addMultipleBrainPairs('training', fsFace, fsBrain, faceWindowSize, ...
- bandpassLowCut, bandpassHighCut, butterOrder, movAvgBrainSec, ignoreInitialSec);
- case 4
- trainingPairs = [];
- disp('Training pairs cleared.');
- case 5
- addMultipleBrainPairs('testing', fsFace, fsBrain, faceWindowSize, ...
- bandpassLowCut, bandpassHighCut, butterOrder, movAvgBrainSec, ignoreInitialSec);
- case 6
- testingPairs = [];
- disp('Testing pairs cleared.');
- case 7
- currentModel = [];
- currentModelType = '';
- disp('Cleared in-memory model.');
- case 8
- [modelFile, modelPath] = uigetfile('*.mat','Select a pre-trained model file');
- if isequal(modelFile, 0)
- disp('No model file selected.');
- else
- loadedData = load(fullfile(modelPath, modelFile));
- modelTypeChoice = menu('Select model type','Neural Network','Ensemble');
- if modelTypeChoice == 1
- if isfield(loadedData, 'net')
- currentModel = loadedData.net;
- currentModelType = 'Neural Network';
- fprintf('Loaded Neural Network model from: %s\n', modelFile);
- else
- error('The file does not contain variable "net".');
- end
- else
- if isfield(loadedData, 'ensModel')
- currentModel = loadedData.ensModel;
- currentModelType = 'Ensemble';
- fprintf('Loaded Ensemble model from: %s\n', modelFile);
- else
- error('The file does not contain variable "ensModel".');
- end
- end
- end
- case 9
- % Train new model
- if isempty(trainingPairs)
- disp('No training pairs available.');
- continue;
- end
- [Xall, Yall] = concatPairs(trainingPairs);
- if isempty(Xall)
- disp('No valid data in training pairs.');
- continue;
- end
- % Train only the Neural Network model
- net = fitnet(nnHiddenNeurons);
- net.trainParam.epochs = nnEpochs;
- net.trainParam.goal = 1e-4;
- [net, ~] = train(net, Xall', Yall');
- YpredNN = net(Xall');
- rmseNN = sqrt(mean((YpredNN' - Yall).^2));
- r2NN = 1 - sum((Yall - YpredNN').^2) / sum((Yall - mean(Yall)).^2);
- fprintf('Neural Net: RMSE=%.4f, R^2=%.4f\n', rmseNN, r2NN);
- % Set neural network as the current model
- currentModel = net;
- currentModelType = 'Neural Network';
- % Save neural network model
- save(fullfile(savePath, [dataLabel '_trainedNeuralNetwork.mat']), 'net');
- disp('Neural Network model saved to disk.');
- case 10
- % Test Model
- if isempty(testingPairs)
- disp('No testing pairs available.');
- continue;
- end
- if isempty(currentModel)
- disp('No model loaded/trained. Please train or load a model first.');
- continue;
- end
- % Debug messages
- disp('Entering Test Model mode...');
- fprintf('Number of testing pairs: %d\n', length(testingPairs));
- testRMSEs = [];
- testR2s = [];
- testFilteredR2s = [];
- testMAEs = [];
- metricsCell = {}; % {BrainFile, Region, Hemisphere, RMSE, R2, Filtered R^2, MAE}
- for iPair = 1:length(testingPairs)
- fprintf('Processing test pair %d of %d...\n', iPair, length(testingPairs));
- Xtest = testingPairs(iPair).X;
- Ytest = testingPairs(iPair).Y;
- if strcmpi(currentModelType, 'Neural Network')
- Ypred = currentModel(Xtest')';
- else
- Ypred = predict(currentModel, Xtest);
- end
- if length(Ypred) == length(Ytest)
- rmseVal = sqrt(mean((Ypred - Ytest).^2));
- r2Val = 1 - sum((Ytest - Ypred).^2) / sum((Ytest - mean(Ytest)).^2);
- else
- minLen = min(length(Ypred), length(Ytest));
- rmseVal = sqrt(mean((Ypred(1:minLen) - Ytest(1:minLen)).^2));
- r2Val = NaN;
- end
- % Compute Winsorized R² (Filtered R²)
- errorValues = abs(Ytest - Ypred);
- threshold = prctile(errorValues, 99); % Ignore top 1% of errors
- validIdx = errorValues < threshold;
- r2Filtered = 1 - sum((Ytest(validIdx) - Ypred(validIdx)).^2) / sum((Ytest(validIdx) - mean(Ytest(validIdx))).^2);
- % Compute MAE (Mean Absolute Error)
- maeVal = mean(abs(Ytest - Ypred));
- testRMSEs(end+1) = rmseVal;
- testR2s(end+1) = r2Val;
- testFilteredR2s(end+1) = r2Filtered;
- testMAEs(end+1) = maeVal;
- regionStr = testingPairs(iPair).region;
- hemisphereStr = testingPairs(iPair).hemisphere;
- brainFile = testingPairs(iPair).brainMAT;
- fprintf('Test Pair %d\n Face: %s\n Brain: %s\n Region: %s, Hemi: %s\n', ...
- iPair, testingPairs(iPair).faceCSV, brainFile, regionStr, hemisphereStr);
- fprintf(' RMSE=%.4f, R^2=%.4f, Filtered R^2=%.4f, MAE=%.4f\n', rmseVal, r2Val, r2Filtered, maeVal);
- % Save metrics for this test pair
- metricsCell(iPair, :) = {brainFile, regionStr, hemisphereStr, rmseVal, r2Val, r2Filtered, maeVal};
- % Generate the prediction figure and save as .fig and .png
- fig = figure('Visible','off','Position',[100 100 1000 600]);
- plot(Ytest, 'g', 'LineWidth', 1.2); hold on;
- plot(Ypred, 'r', 'LineWidth', 1.2);
- legend('Actual Brain', 'Predicted', 'Location', 'best');
- title(sprintf('Brain main(t) Prediction (Auto-Regressive Input)\nRMSE=%.4f, R^2=%.4f', rmseVal, r2Val));
- xlabel('Samples'); ylabel('Amplitude');
- [~, bName, ~] = fileparts(brainFile);
- pngName = sprintf('TestPair%d_%s.png', iPair, bName);
- figName = sprintf('TestPair%d_%s.fig', iPair, bName);
- savefig(fig, fullfile(savePath, figName));
- saveas(fig, fullfile(savePath, pngName));
- % Save test data and figure info in .mat file
- matFileName = fullfile(savePath, sprintf('TestPair%d_%s.mat', iPair, bName));
- save(matFileName, 'Ytest', 'Ypred', 'brainFile', 'regionStr', 'hemisphereStr', 'rmseVal', 'r2Val');
- close(fig);
- end
- if ~isempty(testRMSEs)
- avgRMSE = mean(testRMSEs);
- avgR2 = mean(testR2s);
- fprintf('---\nOverall Test RMSE=%.4f, R^2=%.4f\n', avgRMSE, avgR2);
- end
- header = {'BrainFile', 'Region', 'Hemisphere', 'RMSE', 'R2', 'Filtered R^2', 'MAE'};
- if ~isempty(metricsCell)
- outCell = [header; metricsCell];
- else
- outCell = header;
- end
- outFile = fullfile(savePath, testMetricsFile);
- writecell(outCell, outFile, 'Sheet', 'TestMetrics', 'Range', 'A1');
- fprintf('Wrote test metrics to %s\n', outFile);
- case 11
- % Save pairs to disk
- pairsFile = fullfile(savePath, pairsBackupFile);
- save(pairsFile, 'trainingPairs', 'testingPairs');
- fprintf('Saved pairs to %s\n', pairsFile);
- case 12
- % Load pairs from disk
- pairsFile = fullfile(savePath, pairsBackupFile);
- if exist(pairsFile, 'file')
- load(pairsFile, 'trainingPairs', 'testingPairs');
- fprintf('Loaded pairs from %s\n', pairsFile);
- fprintf('Training pairs: %d, Testing pairs: %d\n', numel(trainingPairs), numel(testingPairs));
- else
- fprintf('No backup file found at %s\n', pairsFile);
- end
- case 13
- disp('Exiting application.');
- break;
- end
- end
- %% Nested Functions
- function addMultipleBrainPairs(modeStr, fsFace, fsBrain, faceWindowSize, bandpassLowCut, bandpassHighCut, bOrder, movAvgBrainSec, ignoreSec)
- [faceFile, facePath] = uigetfile('*.csv', sprintf('Select Face CSV (%.0f Hz) for %s', fsFace, modeStr));
- if isequal(faceFile, 0)
- disp('No face CSV selected.');
- return;
- end
- faceFullPath = fullfile(facePath, faceFile);
- [brainFiles, brainPath] = uigetfile('*.mat', sprintf('Select Brain MAT(s) (%.0f Hz) for %s', fsBrain, modeStr), 'MultiSelect', 'on');
- if isequal(brainFiles, 0)
- disp('No brain MAT selected.');
- return;
- end
- if ischar(brainFiles)
- brainFiles = {brainFiles};
- end
- % Load and aggregate face data using a time-window approach and z-scoring
- [faceTime, faceDataWindowed] = loadFaceCSV_withAggregatesWindowed(faceFullPath, fsFace, faceWindowSize);
- if isempty(faceDataWindowed)
- warning('Face aggregator is empty for %s. Skipping pair addition.', faceFile);
- return;
- end
- for iB = 1:length(brainFiles)
- brainFullPath = fullfile(brainPath, brainFiles{iB});
- % Load brain data and compute additional derived features
- [brainTime, brainMain, optionalBrainFeats] = loadBrainMAT_andDerived(brainFullPath, fsBrain);
- % Align and construct the final input and target.
- [Xaligned, Yaligned] = alignAndConstructAutoReg(faceTime, faceDataWindowed, fsFace, ...
- brainTime, brainMain, optionalBrainFeats, fsBrain, ...
- bandpassLowCut, bandpassHighCut, bOrder, ...
- movAvgBrainSec, ignoreSec);
- % Skip pair if Xaligned is empty or has inconsistent columns.
- if isempty(Xaligned) || size(Xaligned,2) == 0 || isempty(Yaligned)
- fprintf('Skipping pair for Brain file: %s due to insufficient data.\n', brainFiles{iB});
- continue;
- end
- newPair.faceCSV = faceFullPath;
- newPair.brainMAT = brainFullPath;
- % Parse region and hemisphere from file name (simple heuristic)
- [~, baseName, ~] = fileparts(brainFullPath);
- tokens = regexp(baseName, '^(.*?)(bilateral|left|right)', 'tokens', 'once');
- if ~isempty(tokens)
- regionStr = regexprep(tokens{1}, '_$', '');
- hemisphereStr = tokens{2};
- else
- regionStr = baseName;
- hemisphereStr = 'unknown';
- end
- newPair.region = regionStr;
- newPair.hemisphere = hemisphereStr;
- newPair.X = Xaligned;
- newPair.Y = Yaligned;
- if strcmpi(modeStr, 'training')
- trainingPairs = [trainingPairs; newPair];
- fprintf('Added to TRAINING: Face=%s, Brain=%s\n', faceFile, brainFiles{iB});
- else
- testingPairs = [testingPairs; newPair];
- fprintf('Added to TESTING: Face=%s, Brain=%s\n', faceFile, brainFiles{iB});
- end
- end
- end
- function [faceTimeOut, faceDataWindowed] = loadFaceCSV_withAggregatesWindowed(csvFile, fsFace, windowSize)
- opts = detectImportOptions(csvFile, 'VariableNamingRule', 'preserve');
- T = readtable(csvFile, opts);
- % Remove columns with "score" and "track"
- scoreMask = contains(T.Properties.VariableNames, 'score', 'IgnoreCase', true);
- T(:, scoreMask) = [];
- if ismember('track', T.Properties.VariableNames)
- T = removevars(T, 'track');
- end
- % Find frame column among 'Frame','frame','frame_idx'
- possibleFrames = {'Frame', 'frame', 'frame_idx'};
- colFound = intersect(possibleFrames, T.Properties.VariableNames, 'stable');
- if ~isempty(colFound)
- frameVec = T.(colFound{1});
- T = removevars(T, colFound{1});
- else
- if width(T) < 2
- error('No recognized frame column.');
- end
- frameVec = T{:,2};
- T(:,2) = [];
- end
- faceTime = (frameVec - frameVec(1)) / fsFace;
- % Compute aggregator features and velocities
- rawFeatures = computeAggregateFeatures(T); % Nx12
- velFeatures = computeVelocities(rawFeatures, fsFace); % Nx12 => total Nx24
- allFeatures = [rawFeatures, velFeatures];
- [faceTimeU, idxU] = unique(faceTime, 'stable');
- allFeatures = allFeatures(idxU, :);
- faceTime = faceTimeU;
- [faceTimeClean, faceData] = cleanAndInterpolateFaceData(faceTime, allFeatures);
- % Z-score each column
- zFaceData = zscore(faceData, 0, 1);
- % Build time-window aggregator
- [faceTimeOut, faceDataWindowed] = buildTimeWindow(faceTimeClean, zFaceData, windowSize);
- end
- function [timeWin, dataWin] = buildTimeWindow(timeIn, dataIn, wSize)
- N = length(timeIn);
- c = size(dataIn, 2);
- outRows = N - wSize + 1;
- if outRows < 1
- warning('Not enough data for time-window aggregator. Returning empty.');
- timeWin = [];
- dataWin = [];
- return;
- end
- dataWin = zeros(outRows, c * wSize);
- timeWin = timeIn(wSize:end);
- for i = 1:outRows
- block = dataIn(i:i+wSize-1, :); % [wSize x c]
- dataWin(i, :) = block(:)'; % Flatten row-major
- end
- end
- function [brainTime, brainMain, optionalBrainFeats] = loadBrainMAT_andDerived(matFile, fsBrain)
- S = load(matFile);
- fieldNames = fieldnames(S);
- hemoField = '';
- for iF = 1:numel(fieldNames)
- if contains(fieldNames{iF}, 'hemodynamic', 'IgnoreCase', true) && ~contains(fieldNames{iF}, 'sub', 'IgnoreCase', true)
- hemoField = fieldNames{iF};
- break;
- end
- end
- if isempty(hemoField)
- % fallback: look for 'hemo'
- for iF = 1:numel(fieldNames)
- if contains(fieldNames{iF}, 'hemo', 'IgnoreCase', true)
- hemoField = fieldNames{iF};
- break;
- end
- end
- end
- if isempty(hemoField)
- error('No hemodynamic field found in %s.', matFile);
- end
- brainMain = S.(hemoField);
- if isfield(S, 'hemoTime')
- brainTime = S.hemoTime;
- elseif isfield(S, 'time')
- brainTime = S.time;
- else
- nSamples = length(brainMain);
- brainTime = (0:nSamples-1)' / fsBrain;
- end
- % If brainMain is multi-dimensional, use PCA to extract top 3 components.
- if size(brainMain, 2) > 1
- [~, score, ~] = pca(brainMain);
- optionalBrainFeats = score(:, 1:3);
- else
- % Otherwise, compute frequency-domain features using a 1-second window.
- optionalBrainFeats = computeBrainFreqFeatures(brainMain, fsBrain);
- % Interpolate to match length of brainMain if needed.
- if size(optionalBrainFeats, 1) < length(brainMain)
- tFreq = linspace(0, 1, size(optionalBrainFeats, 1))';
- tBrain = linspace(0, 1, length(brainMain))';
- optionalBrainFeats = interp1(tFreq, optionalBrainFeats, tBrain, 'linear', 'extrap');
- end
- end
- end
- function freqFeats = computeBrainFreqFeatures(brainSignal, fs)
- % Compute frequency-domain features using a 1-second window (fs samples)
- winSize = round(1 * fs); % 1-second window
- N = length(brainSignal);
- nOut = N - winSize + 1;
- freqFeats = zeros(nOut, 2); % 2 features: power in two bands
- for i = 1:nOut
- windowData = brainSignal(i:i+winSize-1);
- windowData = windowData .* hamming(winSize);
- X = fft(windowData);
- P2 = abs(X / winSize).^2;
- P1 = P2(1:floor(winSize/2)+1);
- f = (0:floor(winSize/2))' * (fs / winSize);
- % Define two bands: 0.1-0.5 Hz and 0.5-4 Hz (adjust if needed)
- idx1 = f >= 0.1 & f < 1;
- idx2 = f >= 1 & f <= 5;
- power1 = sum(P1(idx1));
- power2 = sum(P1(idx2));
- freqFeats(i, :) = [power1, power2];
- end
- end
- function [Xaligned, Yaligned] = alignAndConstructAutoReg(faceTime, faceDataWindowed, fsFace, ...
- brainTime, brainMainInput, optionalBrainFeats, fsBrain, lowCut, highCut, bOrder, movAvgBrainSec, ignoreSec)
- % 1) Band-pass filter, compute DF/F, smooth
- yFilt = bandpassFilter(brainMainInput, fsBrain, lowCut, highCut, bOrder);
- baseVal = mean(yFilt);
- yDFF = (yFilt / baseVal) - 1;
- windowBrain = round(movAvgBrainSec * fsBrain);
- yProc = smoothdata(yDFF, 'movmean', windowBrain);
- % 2) Remove initial ignoreSec from both face and brain
- faceSkip = round(ignoreSec * fsFace);
- if faceSkip < length(faceTime)
- faceTime(1:faceSkip) = [];
- faceDataWindowed(1:faceSkip, :) = [];
- else
- warning('Ignoring entire face signal? Check data length.');
- faceTime(faceSkip:end) = [];
- faceDataWindowed(faceSkip:end, :) = [];
- end
- brainSkip = round(ignoreSec * fsBrain);
- if brainSkip < length(brainTime)
- brainTime(1:brainSkip) = [];
- yProc(1:brainSkip) = [];
- if ~isempty(optionalBrainFeats)
- optionalBrainFeats(1:brainSkip, :) = [];
- end
- else
- warning('Ignoring entire brain signal? Check data length.');
- brainTime(brainSkip:end) = [];
- yProc(brainSkip:end) = [];
- if ~isempty(optionalBrainFeats)
- optionalBrainFeats(1:brainSkip) = [];
- end
- end
- % 3) Resample face aggregator onto brain time
- faceInterp = zeros(length(brainTime), size(faceDataWindowed, 2));
- for c = 1:size(faceDataWindowed, 2)
- faceInterp(:, c) = interp1(faceTime, faceDataWindowed(:, c), brainTime, 'linear', 'extrap');
- end
- % 4) Resample optional brain features (if any) onto same time
- if ~isempty(optionalBrainFeats)
- brainFeatsInterp = zeros(length(brainTime), size(optionalBrainFeats, 2));
- % We assume optionalBrainFeats covers 0..1 in normalized time
- tOptional = linspace(0, 1, size(optionalBrainFeats, 1))';
- tBrain = linspace(0, 1, length(brainTime))';
- for c = 1:size(optionalBrainFeats, 2)
- brainFeatsInterp(:, c) = interp1(tOptional, optionalBrainFeats(:, c), tBrain, 'linear', 'extrap');
- end
- else
- brainFeatsInterp = [];
- end
- % 5) Construct auto-reg input with BrainMain(t-1)
- yShift = [NaN; yProc(1:end-1)];
- idxValid = 2:length(yProc); % remove first sample
- Xface = faceInterp(idxValid, :);
- Yfinal = yProc(idxValid);
- yShiftValid = yShift(idxValid);
- if ~isempty(brainFeatsInterp)
- bFeats = brainFeatsInterp(idxValid, :);
- Xall = [Xface, bFeats, yShiftValid];
- else
- Xall = [Xface, yShiftValid];
- end
- Xaligned = Xall;
- Yaligned = Yfinal;
- end
- function y = bandpassFilter(x, fs, lowCut, highCut, order)
- nyq = fs / 2;
- [b, a] = butter(order, [lowCut, highCut] / nyq, 'bandpass');
- y = filtfilt(b, a, x);
- end
- function [Xall, Yall] = concatPairs(pairsStruct)
- Xall = [];
- Yall = [];
- for i = 1:length(pairsStruct)
- Xi = pairsStruct(i).X;
- Yi = pairsStruct(i).Y;
- if isempty(Xi) || isempty(Yi)
- fprintf('Pair %d is empty. Skipping.\n', i);
- continue;
- end
- if isempty(Xall)
- Xall = Xi;
- Yall = Yi;
- else
- if size(Xi, 2) ~= size(Xall, 2)
- fprintf('Pair %d has %d columns, expected %d. Skipping pair.\n', i, size(Xi,2), size(Xall,2));
- continue;
- end
- Xall = [Xall; Xi];
- Yall = [Yall; Yi];
- end
- end
- end
- function [faceTime, faceData] = cleanAndInterpolateFaceData(faceTimeIn, faceDataIn)
- cleanTime = faceTimeIn(:);
- cleanData = faceDataIn;
- badMask = ~isfinite(cleanTime);
- if any(badMask)
- cleanTime(badMask) = [];
- cleanData(badMask, :) = [];
- end
- [nr, nc] = size(cleanData);
- for c = 1:nc
- colVals = cleanData(:, c);
- invalidMask = (colVals == 0) | isnan(colVals) | isinf(colVals);
- colVals(invalidMask) = NaN;
- colVals = fillmissing(colVals, 'linear', 'EndValues', 'extrap');
- cleanData(:, c) = colVals;
- end
- faceTime = cleanTime;
- faceData = cleanData;
- end
- function aggFeatures = computeAggregateFeatures(T)
- % This function computes 12 aggregator features from face data table T.
- % Updated to handle the possibility that only 3 whisker points exist
- % instead of 4. It checks columns dynamically.
- % The 12 aggregator features are:
- % (1) eyeCentroid_x
- % (2) eyeCentroid_y
- % (3) eyeArea
- % (4) noseMid_x
- % (5) noseMid_y
- % (6) noseDist
- % (7) mouthMid_x
- % (8) mouthMid_y
- % (9) mouthDist
- % (10) whiskerCentroid_x
- % (11) whiskerCentroid_y
- % (12) whiskerSpread
- N = height(T);
- aggFeatures = zeros(N, 12);
- % Whisker columns that might exist:
- possibleWhiskers = {'Whisker_T_L', 'Whisker_T_R', 'Whisker_B_L', 'Whisker_B_R'};
- for i = 1:N
- %% Eye features (Top, Bottom, Left, Right)
- eX = [safeVal(T, i, 'Top.x'), safeVal(T, i, 'Bottom.x'), safeVal(T, i, 'Left.x'), safeVal(T, i, 'Right.x')];
- eY = [safeVal(T, i, 'Top.y'), safeVal(T, i, 'Bottom.y'), safeVal(T, i, 'Left.y'), safeVal(T, i, 'Right.y')];
- eyeCentroid_x = mean(eX);
- eyeCentroid_y = mean(eY);
- w = max(eX) - min(eX);
- h = max(eY) - min(eY);
- eyeArea = w * h;
- %% Nose features (Nose_Top, Nose_Bottom)
- nxTop = safeVal(T, i, 'Nose_Top.x');
- nyTop = safeVal(T, i, 'Nose_Top.y');
- nxBot = safeVal(T, i, 'Nose_Bottom.x');
- nyBot = safeVal(T, i, 'Nose_Bottom.y');
- noseMid_x = mean([nxTop, nxBot]);
- noseMid_y = mean([nyTop, nyBot]);
- noseDist = sqrt((nxBot - nxTop)^2 + (nyBot - nyTop)^2);
- %% Mouth features (Mouth_Top, Mouth_Bottom)
- mxTop = safeVal(T, i, 'Mouth_Top.x');
- myTop = safeVal(T, i, 'Mouth_Top.y');
- mxBot = safeVal(T, i, 'Mouth_Bottom.x');
- myBot = safeVal(T, i, 'Mouth_Bottom.y');
- mouthMid_x = mean([mxTop, mxBot]);
- mouthMid_y = mean([myTop, myBot]);
- mouthDist = sqrt((mxBot - mxTop)^2 + (myBot - myTop)^2);
- %% Whisker features
- whiskerXvals = [];
- whiskerYvals = [];
- for widx = 1:length(possibleWhiskers)
- baseLabel = possibleWhiskers{widx};
- xLabel = [baseLabel, '.x'];
- yLabel = [baseLabel, '.y'];
- if ismember(xLabel, T.Properties.VariableNames) && ismember(yLabel, T.Properties.VariableNames)
- wX = safeVal(T, i, xLabel);
- wY = safeVal(T, i, yLabel);
- if ~(wX == 0 && wY == 0)
- whiskerXvals(end+1) = wX;
- whiskerYvals(end+1) = wY;
- end
- end
- end
- if isempty(whiskerXvals)
- whiskerCentroid_x = 0;
- whiskerCentroid_y = 0;
- whiskerSpread = 0;
- else
- whiskerCentroid_x = mean(whiskerXvals);
- whiskerCentroid_y = mean(whiskerYvals);
- ww = max(whiskerXvals) - min(whiskerXvals);
- hh = max(whiskerYvals) - min(whiskerYvals);
- whiskerSpread = ww * hh;
- end
- aggFeatures(i, :) = [eyeCentroid_x, eyeCentroid_y, eyeArea, noseMid_x, noseMid_y, noseDist, mouthMid_x, mouthMid_y, mouthDist, whiskerCentroid_x, whiskerCentroid_y, whiskerSpread];
- end
- end
- function val = safeVal(T, row, colName)
- if ismember(colName, T.Properties.VariableNames)
- val = T.(colName)(row);
- else
- val = 0;
- end
- end
- function vel = computeVelocities(features, fs)
- vel = [zeros(1, size(features,2)); diff(features)] * fs;
- end
- end
FaceBrainPredictionGUI_Facemap_6s_6f.m, under CC-BY-4.0 · at the source
Overview
- Department of Neurosciences, Cleveland Clinic Research, Cleveland, OH, USA
- Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, USA
- Department of Biomedical and Chemical Engineering, Cleveland State University, Cleveland, OH, USA
- Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA
- Department of Neurosciences, Case Western Reserve University, Cleveland, OH, USA
- Genomic Sciences and Systems Biology, Cleveland Clinic Research, Cleveland, OH, USA
- Department of Cancer Sciences, Cleveland Clinic Research, Cleveland, OH, USA
- Comprehensive Cancer Center, Case Western Reserve University, Cleveland, OH, USA
- Rose Ella Burkhardt Brain Tumor & Neuro-Oncology Center, Cleveland Clinic, Cleveland, OH, USA
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
- Department of Molecular Medicine, Case Western Reserve University, Cleveland, OH, USA
Abstract
The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.
Repositories
Its files are read in the Code ↔ Paper reader above, with 5 matches between paragraphs and lines of code.
Zenodo 19644866
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
- 27 September 2026: the link answers (HTTP 200)
5 files
- DeepFace_12_Point_Precis
ion.m , MATLAB, 80 lines - FaceBrainPredictionGUI_D
eepFace_6s_6f.m , MATLAB, 697 lines, 1 match - FaceBrainPredictionGUI_D
eepFace_8m_100Hz.m , MATLAB, 551 lines - FaceBrainPredictionGUI_F
acemap_6s_6f.m , MATLAB, 736 lines, 3 matches - FaceBrainPredictionGUI_F
acemap_8m_100Hz.m , MATLAB, 542 lines, 1 match
yyildirimlab/DeepFaceMouse
7ed40947a3ec1a06913a7dc3b33a5155cdf79d8f, 11 June 2026Availability: 1 check, the latest on 27 September 2026: the link answers
- 27 September 2026: the link answers
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 5 scripts, each with its path and the digest of its content;
- 5 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Datasets cited
- zenodo:19635401, at Zenodo; found in “Data and code availability”
- zenodo:19636408, at Zenodo; found in “Data and code availability”
- zenodo:19636758, at Zenodo; found in “Data and code availability”
- zenodo:19636860, at Zenodo; found in “Data and code availability”
- zenodo:19636875, at Zenodo; found in “Data and code availability”
- zenodo:19636972, at Zenodo; found in “Data and code availability”
- zenodo:19636979, at Zenodo; found in “Data and code availability”
- zenodo:19637002, at Zenodo; found in “Data and code availability”
- zenodo:19637463, at Zenodo; found in “Data and code availability”
Code and data availability statement
The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:
- it points to 9 datasets: Zenodo 19635401, Zenodo 19636408, Zenodo 19636758, Zenodo 19636860, Zenodo 19636875, Zenodo 19636972, Zenodo 19636979, Zenodo 19637002, Zenodo 19637463
- it points to the authors' code: yyildirimlab/
DeepFaceMouse , Zenodo 19644866 - it says that the data are available on request
- it says that the code is available on request
Read it in the paper: doi.org/10.1016/j.crmeth.2026.101480.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 2, 28 September 2026
- Authors: added Murat Yildirim (0000-0003-0853-2557); removed Murat Yildirim
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 17 authors, 10 keywords, 6 MeSH terms, 7 funders, 40 references, 6 RRIDs.
Cite
This paper
Ozdemirli, K., Connor, T., Kim, K., Unlu, E. A., Lacin, M. E., Maldonado, M., Bell, F., Leal, T. P., Oksuz, C., Sloan, A., Sarn, N., Chomyk, A., Trapp, B., Lathia, J. D., Mata, I., Eng, C., & Yildirim, M. (2026). DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice. Cell reports methods, 6(8), 101480. https://
BibTeX
@article{ozdemirli2026de
author = {Ozdemirli, Kemal and Connor, Tenesha and Kim, Kaleb and Unlu, Ersan Alp and Lacin, Macit Emre and Maldonado, Miguel and Bell, Frederick and Leal, Thiago Peixoto and Oksuz, Caglar and Sloan, Anthony and Sarn, Nicholas and Chomyk, Anthony and Trapp, Bruce and Lathia, Justin D and Mata, Ignacio and Eng, Charis and Yildirim, Murat},
title = {{DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice}},
journal = {Cell reports methods},
year = {2026},
month = jun,
volume = {6},
number = {8},
pages = {101480},
publisher = {Elsevier},
issn = {2667-2375},
doi = {10.1016/
url = {https://
pmid = {42259296},
pmcid = {PMC13494551}
}
RIS
TY - JOUR
AU - Ozdemirli, Kemal
AU - Connor, Tenesha
AU - Kim, Kaleb
AU - Unlu, Ersan Alp
AU - Lacin, Macit Emre
AU - Maldonado, Miguel
AU - Bell, Frederick
AU - Leal, Thiago Peixoto
AU - Oksuz, Caglar
AU - Sloan, Anthony
AU - Sarn, Nicholas
AU - Chomyk, Anthony
AU - Trapp, Bruce
AU - Lathia, Justin D
AU - Mata, Ignacio
AU - Eng, Charis
AU - Yildirim, Murat
TI - DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice
T2 - Cell reports methods
J2 - Cell Rep Methods
PY - 2026
DA - 2026/
VL - 6
IS - 8
SP - 101480
SN - 2667-2375
PB - Elsevier
DO - 10.1016/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1016/
"type": "article-journal",
"title": "DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice",
"container-title": "Cell reports methods",
"author": [
{
"family": "Ozdemirli",
"given": "Kemal"
},
{
"family": "Connor",
"given": "Tenesha"
},
{
"family": "Kim",
"given": "Kaleb"
},
{
"family": "Unlu",
"given": "Ersan Alp"
},
{
"family": "Lacin",
"given": "Macit Emre"
},
{
"family": "Maldonado",
"given": "Miguel"
},
{
"family": "Bell",
"given": "Frederick"
},
{
"family": "Leal",
"given": "Thiago Peixoto"
},
{
"family": "Oksuz",
"given": "Caglar"
},
{
"family": "Sloan",
"given": "Anthony"
},
{
"family": "Sarn",
"given": "Nicholas"
},
{
"family": "Chomyk",
"given": "Anthony"
},
{
"family": "Trapp",
"given": "Bruce"
},
{
"family": "Lathia",
"given": "Justin D"
},
{
"family": "Mata",
"given": "Ignacio"
},
{
"family": "Eng",
"given": "Charis"
},
{
"family": "Yildirim",
"given": "Murat"
}
],
"container-title-short":
"volume": "6",
"issue": "8",
"page": "101480",
"DOI": "10.1016/
"PMID": "42259296",
"PMCID": "PMC13494551",
"ISSN": "2667-2375",
"publisher": "Elsevier",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
8
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1038/s41593-023-01490-6 [code]
- Facemap: a framework for modeling neural activity based on orofacial tracking.Journal: Nature neuroscienceIn common: mouse, 9 references
- [2] doi:10.1038/s41592-024-02521-1
- ONIX: a unified open-source platform for multimodal neural recording and perturbation during naturalistic behavior.Journal: Nature methodsIn common: methods / tools, mouse, 8 references
- [3] doi:10.7554/elife.79305 [code]
- Open-source tools for behavioral video analysis: Setup, methods, and best practices.Journal: eLifeIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, methods / tools, 7 references
- [4] doi:10.1186/s12915-023-01660-9 [code]
- An automated, low-latency environment for studying the neural basis of behavior in freely moving rats.Journal: BMC biologyIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, 7 references
- [5] doi:10.1038/s41593-026-02262-8 [code]
- Cheese3D enables sensitive detection and analysis of whole-face movement in mice.Journal: Nature neuroscienceIn common: mouse, 7 references
- [6] doi:10.1038/s41592-024-02318-2 [code]
- Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics.Journal: Nature methodsIn common: methods / tools, mouse, 6 references
- [7] doi:10.1126/sciadv.adu9136 [code]
- High-resolution in vivo kinematic tracking with customized injectable fluorescent nanoparticles.Journal: Science advancesIn common: methods / tools, mouse, 6 references
- [8] doi:10.7554/elife.74314 [code]
- BehaviorDEPOT is a simple, flexible tool for automated behavioral detection based on markerless pose tracking.Journal: eLifeIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, methods / tools, mouse, 4 references
- [9] doi:10.1038/s41377-023-01248-6 [code]
- DEEP-squared: deep learning powered De-scattering with Excitation Patterning.Journal: Light, science & applicationsIn common: optical imaging (calcium, voltage, 2-photon), 1 reference, author Murat Yildirim
- [10] doi:10.7554/elife.78079
- Label-free three-photon imaging of intact human cerebral organoids for tracking early events in brain development and deficits in Rett syndrome.Journal: eLifeIn common: optical imaging (calcium, voltage, 2-photon), 1 reference, author Murat Yildirim
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 2 repositories of the authors' code, each at its verified commit and with its license, 5 scripts, and 5 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:efb55a46b1c46bc5…
Add the badge to the code's README
If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.
Markdown
<!-- oscr-badge:start -->
[](https://openscicode.org/badge/doi/10.1016/j.crmeth.2026.101480)
<!-- oscr-badge:end -->reStructuredText
.. oscr-badge:start
.. image:: https://openscicode.org/badge/doi/10.1016/j.crmeth.2026.101480.svg
:target: https://openscicode.org/badge/doi/10.1016/j.crmeth.2026.101480
:alt: OSCR
.. oscr-badge:endHTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1016/j.crmeth.2026.101480"><img src="https://openscicode.org/badge/doi/10.1016/j.crmeth.2026.101480.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".
If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.
Report a mismatch between the code and the paper · Report a code error (signed in)
Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.
Report a reproduction failure (signed in)
Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
