DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice.
The 5 matches
- [1] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_DeepFace_6s_6f.m, lines 509–580 · score 0.68 · auto regressive, processed brain, brain features, vector, linearly, filter
- [2] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_8m_100Hz.m, lines 269–314 · score 0.63 · velocity features, raw feature, window aggregation, frame, Facemap, prediction
- [3] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 379–421 · score 0.62 · velocity features, raw feature, window aggregation, frame, Facemap, prediction
- [4] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 634–721 · score 0.60 · eye features, mouth features, centroid, nose, dynamic, prediction
- [5] § STAR★Methods › Method details › Brain activity prediction using custom Deep Learning Model ↔ FaceBrainPredictionGUI_Facemap_6s_6f.m, lines 1–44 · score 0.57 · corresponding brain, face brain, hemodynamic, FaceMap, Facial, tracking
Paper
Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC
The paper is loaded when this pane is shown.
The authors' code
MATLAB · 736 lines · 29 KB · CC-BY-4.0 · 3 matches
- function FaceBrainPredictionGUI_Facemap_NoEnsemble_winsorizedR2_ko
- % FaceBrainPredictionGUI_Facemap_NoEnsemble_winsorizedR2.m
- %
- % Overview:
- %
- % The GUI includes the following functionalities:
- % 1. Set Data Label: Specify a label for the dataset.
- % 2. Change Save Folder: Select a folder to save outputs, models, graphs, and test data.
- % 3. Add Training Pairs: Select face CSV and corresponding brain MAT files for training.
- % 4. Clear Training Pairs: Remove current training data from memory.
- % 5. Add Testing Pairs: Select face CSV and corresponding brain MAT files for testing.
- % 6. Clear Testing Pairs: Remove current testing data from memory.
- % 7. Clear In-Memory Model: Reset the current model.
- % 8. Load Pre-Trained Model: Load a previously saved model (Neural Network or Ensemble).
- % 9. Train New Model: Train a new Neural Network model using the training pairs.
- % 10. Test Model: Run the model on testing pairs. For each test pair:
- % - Computes performance metrics: RMSE, standard R², Winsorized R² (Filtered R²), and MAE.
- % - Generates a figure comparing actual and predicted signals.
- % - Saves the figure as both .fig and .png files.
- % - Saves test data and metrics in a .mat file.
- % - Writes overall test metrics to an Excel file.
- % 11. Save Pairs to Disk: Save current training/testing pairs to a .mat backup file.
- % 12. Load Pairs from Disk: Load training/testing pairs from a backup file.
- % 13. Exit: Exit the application.
- %
- % Instructions for New Users:
- % - Prepare your face data as CSV files that include columns for tracking facial features.
- % - Prepare your brain data as MAT files containing a variable with hemodynamic signals.
- % The script automatically detects the relevant field (e.g., 'hemodynamic').
- % - Follow the menu options to add training/testing pairs.
- % - Train your model and then test it; the script will generate and save graphs and metrics.
- %
- % Outputs:
- % - Trained model saved as a .mat file.
- % - Test graphs saved as .fig and .png files.
- % - Test data and metrics saved as individual .mat files.
- % - An Excel file summarizing test metrics.
- %
- % Note:
- % - The Winsorized R² (Filtered R²) calculation ignores the top 5% of errors to
- % reduce the impact of outliers.
- clc; clear;
- %% Default Parameters
- dataLabel = 'withaddbrain';
- savePath = pwd; % Default: current folder
- fsBrain = 20; % Brain sampling rate (Hz)
- fsFace = 120; % Face sampling rate (Hz)
- bandpassLowCut = 0.1; % For brain band-pass filtering
- bandpassHighCut = 9.99;
- butterOrder = 2;
- movAvgBrainSec = 1; % 1 s moving average for brain smoothing
- ignoreInitialSec = 5; % Ignore first 5 s of both signals
- faceWindowSize = 5; % Number of face frames per time-window aggregator
- nnHiddenNeurons = 5;
- nnEpochs = 100;
- % ensLearningCycles = 50; % Not used
- % Filenames for saving test metrics and pairs backup
- testMetricsFile = 'TestMetrics.xlsx';
- pairsBackupFile = [dataLabel '_pairsBackup.mat'];
- %% Data in Memory
- trainingPairs = [];
- testingPairs = [];
- %% Model
- currentModel = [];
- currentModelType = ''; % 'Neural Network' or 'Ensemble'
- %% Main Menu Loop
- while true
- choice = menu('Select an option', ...
- '1. Set Data Label', ...
- '2. Change Save Folder', ...
- '3. Add Training Pairs (Face CSV + Brain MAT)', ...
- '4. Clear Training Pairs', ...
- '5. Add Testing Pairs (Face CSV + Brain MAT)', ...
- '6. Clear Testing Pairs', ...
- '7. Clear In-Memory Model', ...
- '8. Load Pre-Trained Model', ...
- '9. Train New Model', ...
- '10. Test Model', ...
- '11. Save Pairs to Disk', ...
- '12. Load Pairs from Disk', ...
- '13. Exit');
- switch choice
- case 1
- prompt = {'Enter data label (e.g. withaddbrain):'};
- dlg_title = 'Set Data Label';
- answer = inputdlg(prompt, dlg_title, 1, {dataLabel});
- if ~isempty(answer)
- dataLabel = answer{1};
- fprintf('Data label set to: %s\n', dataLabel);
- end
- case 2
- newFolder = uigetdir([], 'Select folder for saving outputs/models');
- if isequal(newFolder, 0)
- disp('No folder selected. Save folder unchanged.');
- else
- savePath = newFolder;
- fprintf('Save folder changed to: %s\n', savePath);
- end
- case 3
- addMultipleBrainPairs('training', fsFace, fsBrain, faceWindowSize, ...
- bandpassLowCut, bandpassHighCut, butterOrder, movAvgBrainSec, ignoreInitialSec);
- case 4
- trainingPairs = [];
- disp('Training pairs cleared.');
- case 5
- addMultipleBrainPairs('testing', fsFace, fsBrain, faceWindowSize, ...
- bandpassLowCut, bandpassHighCut, butterOrder, movAvgBrainSec, ignoreInitialSec);
- case 6
- testingPairs = [];
- disp('Testing pairs cleared.');
- case 7
- currentModel = [];
- currentModelType = '';
- disp('Cleared in-memory model.');
- case 8
- [modelFile, modelPath] = uigetfile('*.mat','Select a pre-trained model file');
- if isequal(modelFile, 0)
- disp('No model file selected.');
- else
- loadedData = load(fullfile(modelPath, modelFile));
- modelTypeChoice = menu('Select model type','Neural Network','Ensemble');
- if modelTypeChoice == 1
- if isfield(loadedData, 'net')
- currentModel = loadedData.net;
- currentModelType = 'Neural Network';
- fprintf('Loaded Neural Network model from: %s\n', modelFile);
- else
- error('The file does not contain variable "net".');
- end
- else
- if isfield(loadedData, 'ensModel')
- currentModel = loadedData.ensModel;
- currentModelType = 'Ensemble';
- fprintf('Loaded Ensemble model from: %s\n', modelFile);
- else
- error('The file does not contain variable "ensModel".');
- end
- end
- end
- case 9
- % Train new model
- if isempty(trainingPairs)
- disp('No training pairs available.');
- continue;
- end
- [Xall, Yall] = concatPairs(trainingPairs);
- if isempty(Xall)
- disp('No valid data in training pairs.');
- continue;
- end
- % Train only the Neural Network model
- net = fitnet(nnHiddenNeurons);
- net.trainParam.epochs = nnEpochs;
- net.trainParam.goal = 1e-4;
- [net, ~] = train(net, Xall', Yall');
- YpredNN = net(Xall');
- rmseNN = sqrt(mean((YpredNN' - Yall).^2));
- r2NN = 1 - sum((Yall - YpredNN').^2) / sum((Yall - mean(Yall)).^2);
- fprintf('Neural Net: RMSE=%.4f, R^2=%.4f\n', rmseNN, r2NN);
- % Set neural network as the current model
- currentModel = net;
- currentModelType = 'Neural Network';
- % Save neural network model
- save(fullfile(savePath, [dataLabel '_trainedNeuralNetwork.mat']), 'net');
- disp('Neural Network model saved to disk.');
- case 10
- % Test Model
- if isempty(testingPairs)
- disp('No testing pairs available.');
- continue;
- end
- if isempty(currentModel)
- disp('No model loaded/trained. Please train or load a model first.');
- continue;
- end
- % Debug messages
- disp('Entering Test Model mode...');
- fprintf('Number of testing pairs: %d\n', length(testingPairs));
- testRMSEs = [];
- testR2s = [];
- testFilteredR2s = [];
- testMAEs = [];
- metricsCell = {}; % {BrainFile, Region, Hemisphere, RMSE, R2, Filtered R^2, MAE}
- for iPair = 1:length(testingPairs)
- fprintf('Processing test pair %d of %d...\n', iPair, length(testingPairs));
- Xtest = testingPairs(iPair).X;
- Ytest = testingPairs(iPair).Y;
- if strcmpi(currentModelType, 'Neural Network')
- Ypred = currentModel(Xtest')';
- else
- Ypred = predict(currentModel, Xtest);
- end
- if length(Ypred) == length(Ytest)
- rmseVal = sqrt(mean((Ypred - Ytest).^2));
- r2Val = 1 - sum((Ytest - Ypred).^2) / sum((Ytest - mean(Ytest)).^2);
- else
- minLen = min(length(Ypred), length(Ytest));
- rmseVal = sqrt(mean((Ypred(1:minLen) - Ytest(1:minLen)).^2));
- r2Val = NaN;
- end
- % Compute Winsorized R² (Filtered R²)
- errorValues = abs(Ytest - Ypred);
- threshold = prctile(errorValues, 99); % Ignore top 1% of errors
- validIdx = errorValues < threshold;
- r2Filtered = 1 - sum((Ytest(validIdx) - Ypred(validIdx)).^2) / sum((Ytest(validIdx) - mean(Ytest(validIdx))).^2);
- % Compute MAE (Mean Absolute Error)
- maeVal = mean(abs(Ytest - Ypred));
- testRMSEs(end+1) = rmseVal;
- testR2s(end+1) = r2Val;
- testFilteredR2s(end+1) = r2Filtered;
- testMAEs(end+1) = maeVal;
- regionStr = testingPairs(iPair).region;
- hemisphereStr = testingPairs(iPair).hemisphere;
- brainFile = testingPairs(iPair).brainMAT;
- fprintf('Test Pair %d\n Face: %s\n Brain: %s\n Region: %s, Hemi: %s\n', ...
- iPair, testingPairs(iPair).faceCSV, brainFile, regionStr, hemisphereStr);
- fprintf(' RMSE=%.4f, R^2=%.4f, Filtered R^2=%.4f, MAE=%.4f\n', rmseVal, r2Val, r2Filtered, maeVal);
- % Save metrics for this test pair
- metricsCell(iPair, :) = {brainFile, regionStr, hemisphereStr, rmseVal, r2Val, r2Filtered, maeVal};
- % Generate the prediction figure and save as .fig and .png
- fig = figure('Visible','off','Position',[100 100 1000 600]);
- plot(Ytest, 'g', 'LineWidth', 1.2); hold on;
- plot(Ypred, 'r', 'LineWidth', 1.2);
- legend('Actual Brain', 'Predicted', 'Location', 'best');
- title(sprintf('Brain main(t) Prediction (Auto-Regressive Input)\nRMSE=%.4f, R^2=%.4f', rmseVal, r2Val));
- xlabel('Samples'); ylabel('Amplitude');
- [~, bName, ~] = fileparts(brainFile);
- pngName = sprintf('TestPair%d_%s.png', iPair, bName);
- figName = sprintf('TestPair%d_%s.fig', iPair, bName);
- savefig(fig, fullfile(savePath, figName));
- saveas(fig, fullfile(savePath, pngName));
- % Save test data and figure info in .mat file
- matFileName = fullfile(savePath, sprintf('TestPair%d_%s.mat', iPair, bName));
- save(matFileName, 'Ytest', 'Ypred', 'brainFile', 'regionStr', 'hemisphereStr', 'rmseVal', 'r2Val');
- close(fig);
- end
- if ~isempty(testRMSEs)
- avgRMSE = mean(testRMSEs);
- avgR2 = mean(testR2s);
- fprintf('---\nOverall Test RMSE=%.4f, R^2=%.4f\n', avgRMSE, avgR2);
- end
- header = {'BrainFile', 'Region', 'Hemisphere', 'RMSE', 'R2', 'Filtered R^2', 'MAE'};
- if ~isempty(metricsCell)
- outCell = [header; metricsCell];
- else
- outCell = header;
- end
- outFile = fullfile(savePath, testMetricsFile);
- writecell(outCell, outFile, 'Sheet', 'TestMetrics', 'Range', 'A1');
- fprintf('Wrote test metrics to %s\n', outFile);
- case 11
- % Save pairs to disk
- pairsFile = fullfile(savePath, pairsBackupFile);
- save(pairsFile, 'trainingPairs', 'testingPairs');
- fprintf('Saved pairs to %s\n', pairsFile);
- case 12
- % Load pairs from disk
- pairsFile = fullfile(savePath, pairsBackupFile);
- if exist(pairsFile, 'file')
- load(pairsFile, 'trainingPairs', 'testingPairs');
- fprintf('Loaded pairs from %s\n', pairsFile);
- fprintf('Training pairs: %d, Testing pairs: %d\n', numel(trainingPairs), numel(testingPairs));
- else
- fprintf('No backup file found at %s\n', pairsFile);
- end
- case 13
- disp('Exiting application.');
- break;
- end
- end
- %% Nested Functions
- function addMultipleBrainPairs(modeStr, fsFace, fsBrain, faceWindowSize, bandpassLowCut, bandpassHighCut, bOrder, movAvgBrainSec, ignoreSec)
- [faceFile, facePath] = uigetfile('*.csv', sprintf('Select Face CSV (%.0f Hz) for %s', fsFace, modeStr));
- if isequal(faceFile, 0)
- disp('No face CSV selected.');
- return;
- end
- faceFullPath = fullfile(facePath, faceFile);
- [brainFiles, brainPath] = uigetfile('*.mat', sprintf('Select Brain MAT(s) (%.0f Hz) for %s', fsBrain, modeStr), 'MultiSelect', 'on');
- if isequal(brainFiles, 0)
- disp('No brain MAT selected.');
- return;
- end
- if ischar(brainFiles)
- brainFiles = {brainFiles};
- end
- % Load and aggregate face data using a time-window approach and z-scoring
- [faceTime, faceDataWindowed] = loadFaceCSV_withAggregatesWindowed(faceFullPath, fsFace, faceWindowSize);
- if isempty(faceDataWindowed)
- warning('Face aggregator is empty for %s. Skipping pair addition.', faceFile);
- return;
- end
- for iB = 1:length(brainFiles)
- brainFullPath = fullfile(brainPath, brainFiles{iB});
- % Load brain data and compute additional derived features
- [brainTime, brainMain, optionalBrainFeats] = loadBrainMAT_andDerived(brainFullPath, fsBrain);
- % Align and construct the final input and target.
- [Xaligned, Yaligned] = alignAndConstructAutoReg(faceTime, faceDataWindowed, fsFace, ...
- brainTime, brainMain, optionalBrainFeats, fsBrain, ...
- bandpassLowCut, bandpassHighCut, bOrder, ...
- movAvgBrainSec, ignoreSec);
- % Skip pair if Xaligned is empty or has inconsistent columns.
- if isempty(Xaligned) || size(Xaligned,2) == 0 || isempty(Yaligned)
- fprintf('Skipping pair for Brain file: %s due to insufficient data.\n', brainFiles{iB});
- continue;
- end
- newPair.faceCSV = faceFullPath;
- newPair.brainMAT = brainFullPath;
- % Parse region and hemisphere from file name (simple heuristic)
- [~, baseName, ~] = fileparts(brainFullPath);
- tokens = regexp(baseName, '^(.*?)(bilateral|left|right)', 'tokens', 'once');
- if ~isempty(tokens)
- regionStr = regexprep(tokens{1}, '_$', '');
- hemisphereStr = tokens{2};
- else
- regionStr = baseName;
- hemisphereStr = 'unknown';
- end
- newPair.region = regionStr;
- newPair.hemisphere = hemisphereStr;
- newPair.X = Xaligned;
- newPair.Y = Yaligned;
- if strcmpi(modeStr, 'training')
- trainingPairs = [trainingPairs; newPair];
- fprintf('Added to TRAINING: Face=%s, Brain=%s\n', faceFile, brainFiles{iB});
- else
- testingPairs = [testingPairs; newPair];
- fprintf('Added to TESTING: Face=%s, Brain=%s\n', faceFile, brainFiles{iB});
- end
- end
- end
- function [faceTimeOut, faceDataWindowed] = loadFaceCSV_withAggregatesWindowed(csvFile, fsFace, windowSize)
- opts = detectImportOptions(csvFile, 'VariableNamingRule', 'preserve');
- T = readtable(csvFile, opts);
- % Remove columns with "score" and "track"
- scoreMask = contains(T.Properties.VariableNames, 'score', 'IgnoreCase', true);
- T(:, scoreMask) = [];
- if ismember('track', T.Properties.VariableNames)
- T = removevars(T, 'track');
- end
- % Find frame column among 'Frame','frame','frame_idx'
- possibleFrames = {'Frame', 'frame', 'frame_idx'};
- colFound = intersect(possibleFrames, T.Properties.VariableNames, 'stable');
- if ~isempty(colFound)
- frameVec = T.(colFound{1});
- T = removevars(T, colFound{1});
- else
- if width(T) < 2
- error('No recognized frame column.');
- end
- frameVec = T{:,2};
- T(:,2) = [];
- end
- faceTime = (frameVec - frameVec(1)) / fsFace;
- % Compute aggregator features and velocities
- rawFeatures = computeAggregateFeatures(T); % Nx12
- velFeatures = computeVelocities(rawFeatures, fsFace); % Nx12 => total Nx24
- allFeatures = [rawFeatures, velFeatures];
- [faceTimeU, idxU] = unique(faceTime, 'stable');
- allFeatures = allFeatures(idxU, :);
- faceTime = faceTimeU;
- [faceTimeClean, faceData] = cleanAndInterpolateFaceData(faceTime, allFeatures);
- % Z-score each column
- zFaceData = zscore(faceData, 0, 1);
- % Build time-window aggregator
- [faceTimeOut, faceDataWindowed] = buildTimeWindow(faceTimeClean, zFaceData, windowSize);
- end
- function [timeWin, dataWin] = buildTimeWindow(timeIn, dataIn, wSize)
- N = length(timeIn);
- c = size(dataIn, 2);
- outRows = N - wSize + 1;
- if outRows < 1
- warning('Not enough data for time-window aggregator. Returning empty.');
- timeWin = [];
- dataWin = [];
- return;
- end
- dataWin = zeros(outRows, c * wSize);
- timeWin = timeIn(wSize:end);
- for i = 1:outRows
- block = dataIn(i:i+wSize-1, :); % [wSize x c]
- dataWin(i, :) = block(:)'; % Flatten row-major
- end
- end
- function [brainTime, brainMain, optionalBrainFeats] = loadBrainMAT_andDerived(matFile, fsBrain)
- S = load(matFile);
- fieldNames = fieldnames(S);
- hemoField = '';
- for iF = 1:numel(fieldNames)
- if contains(fieldNames{iF}, 'hemodynamic', 'IgnoreCase', true) && ~contains(fieldNames{iF}, 'sub', 'IgnoreCase', true)
- hemoField = fieldNames{iF};
- break;
- end
- end
- if isempty(hemoField)
- % fallback: look for 'hemo'
- for iF = 1:numel(fieldNames)
- if contains(fieldNames{iF}, 'hemo', 'IgnoreCase', true)
- hemoField = fieldNames{iF};
- break;
- end
- end
- end
- if isempty(hemoField)
- error('No hemodynamic field found in %s.', matFile);
- end
- brainMain = S.(hemoField);
- if isfield(S, 'hemoTime')
- brainTime = S.hemoTime;
- elseif isfield(S, 'time')
- brainTime = S.time;
- else
- nSamples = length(brainMain);
- brainTime = (0:nSamples-1)' / fsBrain;
- end
- % If brainMain is multi-dimensional, use PCA to extract top 3 components.
- if size(brainMain, 2) > 1
- [~, score, ~] = pca(brainMain);
- optionalBrainFeats = score(:, 1:3);
- else
- % Otherwise, compute frequency-domain features using a 1-second window.
- optionalBrainFeats = computeBrainFreqFeatures(brainMain, fsBrain);
- % Interpolate to match length of brainMain if needed.
- if size(optionalBrainFeats, 1) < length(brainMain)
- tFreq = linspace(0, 1, size(optionalBrainFeats, 1))';
- tBrain = linspace(0, 1, length(brainMain))';
- optionalBrainFeats = interp1(tFreq, optionalBrainFeats, tBrain, 'linear', 'extrap');
- end
- end
- end
- function freqFeats = computeBrainFreqFeatures(brainSignal, fs)
- % Compute frequency-domain features using a 1-second window (fs samples)
- winSize = round(1 * fs); % 1-second window
- N = length(brainSignal);
- nOut = N - winSize + 1;
- freqFeats = zeros(nOut, 2); % 2 features: power in two bands
- for i = 1:nOut
- windowData = brainSignal(i:i+winSize-1);
- windowData = windowData .* hamming(winSize);
- X = fft(windowData);
- P2 = abs(X / winSize).^2;
- P1 = P2(1:floor(winSize/2)+1);
- f = (0:floor(winSize/2))' * (fs / winSize);
- % Define two bands: 0.1-0.5 Hz and 0.5-4 Hz (adjust if needed)
- idx1 = f >= 0.1 & f < 1;
- idx2 = f >= 1 & f <= 5;
- power1 = sum(P1(idx1));
- power2 = sum(P1(idx2));
- freqFeats(i, :) = [power1, power2];
- end
- end
- function [Xaligned, Yaligned] = alignAndConstructAutoReg(faceTime, faceDataWindowed, fsFace, ...
- brainTime, brainMainInput, optionalBrainFeats, fsBrain, lowCut, highCut, bOrder, movAvgBrainSec, ignoreSec)
- % 1) Band-pass filter, compute DF/F, smooth
- yFilt = bandpassFilter(brainMainInput, fsBrain, lowCut, highCut, bOrder);
- baseVal = mean(yFilt);
- yDFF = (yFilt / baseVal) - 1;
- windowBrain = round(movAvgBrainSec * fsBrain);
- yProc = smoothdata(yDFF, 'movmean', windowBrain);
- % 2) Remove initial ignoreSec from both face and brain
- faceSkip = round(ignoreSec * fsFace);
- if faceSkip < length(faceTime)
- faceTime(1:faceSkip) = [];
- faceDataWindowed(1:faceSkip, :) = [];
- else
- warning('Ignoring entire face signal? Check data length.');
- faceTime(faceSkip:end) = [];
- faceDataWindowed(faceSkip:end, :) = [];
- end
- brainSkip = round(ignoreSec * fsBrain);
- if brainSkip < length(brainTime)
- brainTime(1:brainSkip) = [];
- yProc(1:brainSkip) = [];
- if ~isempty(optionalBrainFeats)
- optionalBrainFeats(1:brainSkip, :) = [];
- end
- else
- warning('Ignoring entire brain signal? Check data length.');
- brainTime(brainSkip:end) = [];
- yProc(brainSkip:end) = [];
- if ~isempty(optionalBrainFeats)
- optionalBrainFeats(1:brainSkip) = [];
- end
- end
- % 3) Resample face aggregator onto brain time
- faceInterp = zeros(length(brainTime), size(faceDataWindowed, 2));
- for c = 1:size(faceDataWindowed, 2)
- faceInterp(:, c) = interp1(faceTime, faceDataWindowed(:, c), brainTime, 'linear', 'extrap');
- end
- % 4) Resample optional brain features (if any) onto same time
- if ~isempty(optionalBrainFeats)
- brainFeatsInterp = zeros(length(brainTime), size(optionalBrainFeats, 2));
- % We assume optionalBrainFeats covers 0..1 in normalized time
- tOptional = linspace(0, 1, size(optionalBrainFeats, 1))';
- tBrain = linspace(0, 1, length(brainTime))';
- for c = 1:size(optionalBrainFeats, 2)
- brainFeatsInterp(:, c) = interp1(tOptional, optionalBrainFeats(:, c), tBrain, 'linear', 'extrap');
- end
- else
- brainFeatsInterp = [];
- end
- % 5) Construct auto-reg input with BrainMain(t-1)
- yShift = [NaN; yProc(1:end-1)];
- idxValid = 2:length(yProc); % remove first sample
- Xface = faceInterp(idxValid, :);
- Yfinal = yProc(idxValid);
- yShiftValid = yShift(idxValid);
- if ~isempty(brainFeatsInterp)
- bFeats = brainFeatsInterp(idxValid, :);
- Xall = [Xface, bFeats, yShiftValid];
- else
- Xall = [Xface, yShiftValid];
- end
- Xaligned = Xall;
- Yaligned = Yfinal;
- end
- function y = bandpassFilter(x, fs, lowCut, highCut, order)
- nyq = fs / 2;
- [b, a] = butter(order, [lowCut, highCut] / nyq, 'bandpass');
- y = filtfilt(b, a, x);
- end
- function [Xall, Yall] = concatPairs(pairsStruct)
- Xall = [];
- Yall = [];
- for i = 1:length(pairsStruct)
- Xi = pairsStruct(i).X;
- Yi = pairsStruct(i).Y;
- if isempty(Xi) || isempty(Yi)
- fprintf('Pair %d is empty. Skipping.\n', i);
- continue;
- end
- if isempty(Xall)
- Xall = Xi;
- Yall = Yi;
- else
- if size(Xi, 2) ~= size(Xall, 2)
- fprintf('Pair %d has %d columns, expected %d. Skipping pair.\n', i, size(Xi,2), size(Xall,2));
- continue;
- end
- Xall = [Xall; Xi];
- Yall = [Yall; Yi];
- end
- end
- end
- function [faceTime, faceData] = cleanAndInterpolateFaceData(faceTimeIn, faceDataIn)
- cleanTime = faceTimeIn(:);
- cleanData = faceDataIn;
- badMask = ~isfinite(cleanTime);
- if any(badMask)
- cleanTime(badMask) = [];
- cleanData(badMask, :) = [];
- end
- [nr, nc] = size(cleanData);
- for c = 1:nc
- colVals = cleanData(:, c);
- invalidMask = (colVals == 0) | isnan(colVals) | isinf(colVals);
- colVals(invalidMask) = NaN;
- colVals = fillmissing(colVals, 'linear', 'EndValues', 'extrap');
- cleanData(:, c) = colVals;
- end
- faceTime = cleanTime;
- faceData = cleanData;
- end
- function aggFeatures = computeAggregateFeatures(T)
- % This function computes 12 aggregator features from face data table T.
- % Updated to handle the possibility that only 3 whisker points exist
- % instead of 4. It checks columns dynamically.
- % The 12 aggregator features are:
- % (1) eyeCentroid_x
- % (2) eyeCentroid_y
- % (3) eyeArea
- % (4) noseMid_x
- % (5) noseMid_y
- % (6) noseDist
- % (7) mouthMid_x
- % (8) mouthMid_y
- % (9) mouthDist
- % (10) whiskerCentroid_x
- % (11) whiskerCentroid_y
- % (12) whiskerSpread
- N = height(T);
- aggFeatures = zeros(N, 12);
- % Whisker columns that might exist:
- possibleWhiskers = {'Whisker_T_L', 'Whisker_T_R', 'Whisker_B_L', 'Whisker_B_R'};
- for i = 1:N
- %% Eye features (Top, Bottom, Left, Right)
- eX = [safeVal(T, i, 'Top.x'), safeVal(T, i, 'Bottom.x'), safeVal(T, i, 'Left.x'), safeVal(T, i, 'Right.x')];
- eY = [safeVal(T, i, 'Top.y'), safeVal(T, i, 'Bottom.y'), safeVal(T, i, 'Left.y'), safeVal(T, i, 'Right.y')];
- eyeCentroid_x = mean(eX);
- eyeCentroid_y = mean(eY);
- w = max(eX) - min(eX);
- h = max(eY) - min(eY);
- eyeArea = w * h;
- %% Nose features (Nose_Top, Nose_Bottom)
- nxTop = safeVal(T, i, 'Nose_Top.x');
- nyTop = safeVal(T, i, 'Nose_Top.y');
- nxBot = safeVal(T, i, 'Nose_Bottom.x');
- nyBot = safeVal(T, i, 'Nose_Bottom.y');
- noseMid_x = mean([nxTop, nxBot]);
- noseMid_y = mean([nyTop, nyBot]);
- noseDist = sqrt((nxBot - nxTop)^2 + (nyBot - nyTop)^2);
- %% Mouth features (Mouth_Top, Mouth_Bottom)
- mxTop = safeVal(T, i, 'Mouth_Top.x');
- myTop = safeVal(T, i, 'Mouth_Top.y');
- mxBot = safeVal(T, i, 'Mouth_Bottom.x');
- myBot = safeVal(T, i, 'Mouth_Bottom.y');
- mouthMid_x = mean([mxTop, mxBot]);
- mouthMid_y = mean([myTop, myBot]);
- mouthDist = sqrt((mxBot - mxTop)^2 + (myBot - myTop)^2);
- %% Whisker features
- whiskerXvals = [];
- whiskerYvals = [];
- for widx = 1:length(possibleWhiskers)
- baseLabel = possibleWhiskers{widx};
- xLabel = [baseLabel, '.x'];
- yLabel = [baseLabel, '.y'];
- if ismember(xLabel, T.Properties.VariableNames) && ismember(yLabel, T.Properties.VariableNames)
- wX = safeVal(T, i, xLabel);
- wY = safeVal(T, i, yLabel);
- if ~(wX == 0 && wY == 0)
- whiskerXvals(end+1) = wX;
- whiskerYvals(end+1) = wY;
- end
- end
- end
- if isempty(whiskerXvals)
- whiskerCentroid_x = 0;
- whiskerCentroid_y = 0;
- whiskerSpread = 0;
- else
- whiskerCentroid_x = mean(whiskerXvals);
- whiskerCentroid_y = mean(whiskerYvals);
- ww = max(whiskerXvals) - min(whiskerXvals);
- hh = max(whiskerYvals) - min(whiskerYvals);
- whiskerSpread = ww * hh;
- end
- aggFeatures(i, :) = [eyeCentroid_x, eyeCentroid_y, eyeArea, noseMid_x, noseMid_y, noseDist, mouthMid_x, mouthMid_y, mouthDist, whiskerCentroid_x, whiskerCentroid_y, whiskerSpread];
- end
- end
- function val = safeVal(T, row, colName)
- if ismember(colName, T.Properties.VariableNames)
- val = T.(colName)(row);
- else
- val = 0;
- end
- end
- function vel = computeVelocities(features, fs)
- vel = [zeros(1, size(features,2)); diff(features)] * fs;
- end
- end
FaceBrainPredictionGUI_Facemap_6s_6f.m, under CC-BY-4.0 · at the source
Overview
- Department of Neurosciences, Cleveland Clinic Research, Cleveland, OH, USA
- Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, USA
- Department of Biomedical and Chemical Engineering, Cleveland State University, Cleveland, OH, USA
- Department of Computer and Data Sciences, Case Western Reserve University, Cleveland, OH, USA
- Department of Neurosciences, Case Western Reserve University, Cleveland, OH, USA
- Genomic Sciences and Systems Biology, Cleveland Clinic Research, Cleveland, OH, USA
- Department of Cancer Sciences, Cleveland Clinic Research, Cleveland, OH, USA
- Comprehensive Cancer Center, Case Western Reserve University, Cleveland, OH, USA
- Rose Ella Burkhardt Brain Tumor & Neuro-Oncology Center, Cleveland Clinic, Cleveland, OH, USA
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
- Department of Molecular Medicine, Case Western Reserve University, Cleveland, OH, USA
Abstract
The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.
Repositories
Its files are read in the Code ↔ Paper reader above, with 5 matches between paragraphs and lines of code.
Zenodo 19644866
Availability: 1 check, the latest on 27 September 2026: the link answers (HTTP 200)
- 27 September 2026: the link answers (HTTP 200)
5 files
- DeepFace_12_Point_Precis
ion.m , MATLAB, 80 lines - FaceBrainPredictionGUI_D
eepFace_6s_6f.m , MATLAB, 697 lines, 1 match - FaceBrainPredictionGUI_D
eepFace_8m_100Hz.m , MATLAB, 551 lines - FaceBrainPredictionGUI_F
acemap_6s_6f.m , MATLAB, 736 lines, 3 matches - FaceBrainPredictionGUI_F
acemap_8m_100Hz.m , MATLAB, 542 lines, 1 match
yyildirimlab/DeepFaceMouse
7ed40947a3ec1a06913a7dc3b33a5155cdf79d8f, 11 June 2026Availability: 1 check, the latest on 27 September 2026: the link answers
- 27 September 2026: the link answers
The paper's code and data availability statement is in the Data section.
Tracing map
Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.
What the map holds:
- 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
- 5 scripts, each with its path and the digest of its content;
- 5 matches between paragraphs of the paper and lines of the code (method lexical-v1);
- neither the text of the paper nor the code itself.
Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.
Data
Datasets cited
- zenodo:19635401, at Zenodo; found in “Data and code availability”
- zenodo:19636408, at Zenodo; found in “Data and code availability”
- zenodo:19636758, at Zenodo; found in “Data and code availability”
- zenodo:19636860, at Zenodo; found in “Data and code availability”
- zenodo:19636875, at Zenodo; found in “Data and code availability”
- zenodo:19636972, at Zenodo; found in “Data and code availability”
- zenodo:19636979, at Zenodo; found in “Data and code availability”
- zenodo:19637002, at Zenodo; found in “Data and code availability”
- zenodo:19637463, at Zenodo; found in “Data and code availability”
Code and data availability statement
The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:
- it points to 9 datasets: Zenodo 19635401, Zenodo 19636408, Zenodo 19636758, Zenodo 19636860, Zenodo 19636875, Zenodo 19636972, Zenodo 19636979, Zenodo 19637002, Zenodo 19637463
- it points to the authors' code: yyildirimlab/
DeepFaceMouse , Zenodo 19644866 - it says that the data are available on request
- it says that the code is available on request
Read it in the paper: doi.org/10.1016/j.crmeth.2026.101480.
Versions
The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.
Version 2, 28 September 2026
- Authors: added Murat Yildirim (0000-0003-0853-2557); removed Murat Yildirim
Version 1, 27 September 2026: the first record
Recorded: type, language, journal, volume, issue, pages, dates, 17 authors, 10 keywords, 6 MeSH terms, 7 funders, 40 references, 6 RRIDs.
Cite
This paper
Ozdemirli, K., Connor, T., Kim, K., Unlu, E. A., Lacin, M. E., Maldonado, M., Bell, F., Leal, T. P., Oksuz, C., Sloan, A., Sarn, N., Chomyk, A., Trapp, B., Lathia, J. D., Mata, I., Eng, C., & Yildirim, M. (2026). DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice. Cell reports methods, 6(8), 101480. https://
BibTeX
@article{ozdemirli2026de
author = {Ozdemirli, Kemal and Connor, Tenesha and Kim, Kaleb and Unlu, Ersan Alp and Lacin, Macit Emre and Maldonado, Miguel and Bell, Frederick and Leal, Thiago Peixoto and Oksuz, Caglar and Sloan, Anthony and Sarn, Nicholas and Chomyk, Anthony and Trapp, Bruce and Lathia, Justin D and Mata, Ignacio and Eng, Charis and Yildirim, Murat},
title = {{DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice}},
journal = {Cell reports methods},
year = {2026},
month = jun,
volume = {6},
number = {8},
pages = {101480},
publisher = {Elsevier},
issn = {2667-2375},
doi = {10.1016/
url = {https://
pmid = {42259296},
pmcid = {PMC13494551}
}
RIS
TY - JOUR
AU - Ozdemirli, Kemal
AU - Connor, Tenesha
AU - Kim, Kaleb
AU - Unlu, Ersan Alp
AU - Lacin, Macit Emre
AU - Maldonado, Miguel
AU - Bell, Frederick
AU - Leal, Thiago Peixoto
AU - Oksuz, Caglar
AU - Sloan, Anthony
AU - Sarn, Nicholas
AU - Chomyk, Anthony
AU - Trapp, Bruce
AU - Lathia, Justin D
AU - Mata, Ignacio
AU - Eng, Charis
AU - Yildirim, Murat
TI - DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice
T2 - Cell reports methods
J2 - Cell Rep Methods
PY - 2026
DA - 2026/
VL - 6
IS - 8
SP - 101480
SN - 2667-2375
PB - Elsevier
DO - 10.1016/
UR - https://
LA - en
ER -
CSL-JSON
{
"id": "10.1016/
"type": "article-journal",
"title": "DeepFaceMouse enables scalable prediction of large-scale brain activity from facial dynamics in mice",
"container-title": "Cell reports methods",
"author": [
{
"family": "Ozdemirli",
"given": "Kemal"
},
{
"family": "Connor",
"given": "Tenesha"
},
{
"family": "Kim",
"given": "Kaleb"
},
{
"family": "Unlu",
"given": "Ersan Alp"
},
{
"family": "Lacin",
"given": "Macit Emre"
},
{
"family": "Maldonado",
"given": "Miguel"
},
{
"family": "Bell",
"given": "Frederick"
},
{
"family": "Leal",
"given": "Thiago Peixoto"
},
{
"family": "Oksuz",
"given": "Caglar"
},
{
"family": "Sloan",
"given": "Anthony"
},
{
"family": "Sarn",
"given": "Nicholas"
},
{
"family": "Chomyk",
"given": "Anthony"
},
{
"family": "Trapp",
"given": "Bruce"
},
{
"family": "Lathia",
"given": "Justin D"
},
{
"family": "Mata",
"given": "Ignacio"
},
{
"family": "Eng",
"given": "Charis"
},
{
"family": "Yildirim",
"given": "Murat"
}
],
"container-title-short":
"volume": "6",
"issue": "8",
"page": "101480",
"DOI": "10.1016/
"PMID": "42259296",
"PMCID": "PMC13494551",
"ISSN": "2667-2375",
"publisher": "Elsevier",
"URL": "https://
"language": "en",
"issued": {
"date-parts": [
[
2026,
6,
8
]
]
}
}
The tracing map gets a citation of its own once an author has validated it and it has a DOI.
Similar papers
The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.
- [1] doi:10.1038/s41593-026-02262-8 [code]
- Cheese3D enables sensitive detection and analysis of whole-face movement in mice.Journal: Nature neuroscienceIn common: mouse, 7 references
- [2] doi:10.1038/s41467-026-75347-4 [code]
- Sleep reveals dynamics integrating and segregating movement and stimulus representations in V1.Journal: Nature communicationsIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 3 references
- [3] doi:10.1038/s41467-026-75492-w [code]
- Place and behavioral modulation of hippocampal neurons during immobility.Journal: Nature communicationsIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 3 references
- [4] doi:10.1038/s41467-026-76581-6 [code]
- Thalamocortical bursts encode reward contingencies and drive associative learning.Journal: Nature communicationsIn common: Deep Learning Toolbox, Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 1 reference
- [5] doi:10.3389/fnins.2026.1790603 [code]
- A novel behavioral paradigm using mice to study predictive postural control.Journal: Frontiers in neuroscienceIn common: mouse, 4 references
- [6] doi:10.7554/elife.109240 [code]
- Neural activity profiles reveal overlapping, intermingled subpopulations spanning area borders in mouse sensorimotor cortex.Journal: eLifeIn common: Statistics and Machine Learning Toolbox, optical imaging (calcium, voltage, 2-photon), mouse, 3 references
- [7] doi:10.1038/s41467-026-73622-y [code]
- Contextual gating of whisker-evoked responses by frontal cortex supports flexible decision making.Journal: Nature communicationsIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 3 references
- [8] doi:10.1038/s41467-026-71664-w [code]
- Dorsal prefrontal cortex drives perseverative behavior in mice.Journal: Nature communicationsIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 2 references
- [9] doi:10.1038/s41598-026-52322-z [code]
- A transparent wheel-based platform for locomotion-on-demand and multi-view body and facial kinematics in head-fixed mice.Journal: Scientific reportsIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, methods / tools, mouse, 2 references
- [10] doi:10.3389/fendo.2026.1828487 [code]
- Castration-induced nigrostriatal deficits are linked to reduced TrkB and loss of mature spines in the dorsal striatum.Journal: Frontiers in endocrinologyIn common: Signal Processing Toolbox, Statistics and Machine Learning Toolbox, mouse, 2 references
Contribute
The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.
Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.
Claim this paper
Correct its record
Say what each link of this record is, remove the ones that are not the paper's, add the ones that are missing. The correction becomes a new version of the record, in its Versions section.
Validate its tracing map
You validate the map as this page shows it: 2 repositories of the authors' code, each at its verified commit and with its license, 5 scripts, and 5 matches between paragraphs and code (see the Code and Map sections). It then receives a DOI on Zenodo, with you (your ORCID iD) and OSCR as its creators; the code itself is not deposited.
The map's fingerprint: sha256:573031f5f01fcba2…
Add the badge to its README
The badge links the code to this page. Copy one of these into the README of the paper's code: only you decide where it goes, and nothing is changed for you.
Markdown
[, paste the snippet at the top, then “Commit changes…” and, to review it first, “Create a new branch and start a pull request”. You open the pull request; OSCR asks for no permission.
Request its removal
To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).
Discussion, reproductions, activity
Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.
Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.
Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.
