OSCR

SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes.

Code ↔ Paper

14 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 14 matches
  1. [1] § Methodology › Overview of the SpeechDETECT architecture › Component 2.2. cepstral coefficients and spectral features ↔ speechdetect/AcousticFeatureExtractor.py, lines 136–214 · score 0.78 · Spectral Centroid, Cepstral Coefficients, Spectral features, LTAS, Mel, spectrum
  2. [2] § Methodology › Overview of the SpeechDETECT architecture › Component 2.4. loudness and intensity ↔ speechdetect/AcousticFeatureExtractor.py, lines 306–353 · score 0.70 · Sound Pressure, squared amplitude, SPL, RMS, loudness, Root
  3. [3] § Methodology › Overview of the SpeechDETECT architecture › Component 3- computing acoustic and temporal features ↔ speechdetect/acoustic_features/speech_fluency_and_speech_production_dynamics.py, lines 407–421 · score 0.68 · syllabic interval duration, production dynamics, syllable, phoneme, word, fluency
  4. [4] § Methodology › Overview of the SpeechDETECT architecture › Component 2.3. voice quality ↔ speechdetect/acoustic_features/voice_quality.py, lines 244–335 · score 0.65 · Harmonic Ratio, Voice Quality, Noise, NHR, consecutive, sound
  5. [5] § Methodology › Overview of the SpeechDETECT architecture › Component 2.2. cepstral coefficients and spectral features ↔ speechdetect/acoustic_features/cepstral_coefficients_and_spectral_features.py, lines 36–55 · score 0.59 · Cepstral Coefficients, Spectral features, LTAS, spectrum
  6. [6] § Methodology › Overview of the SpeechDETECT architecture › Component 2.8. speech production dynamics ↔ speechdetect/acoustic_features/speech_fluency_and_speech_production_dynamics.py, lines 588–602 · score 0.59 · Relative Sentence Duration, Speech production dynamics, fluency
  7. [7] § Methodology › Overview of the SpeechDETECT architecture › Component 2–voice analysis framework: characterizing vocal traits and temporal dynamics ↔ speechdetect/acoustic_features/speech_fluency_and_speech_production_dynamics.py, lines 24–27 · score 0.58 · speech production dynamics, speech fluency
  8. [8] § Methodology › Overview of the SpeechDETECT architecture ↔ speechdetect/AcousticFeatureExtractor.py, lines 1185–1310 · score 0.57 · fundamental frequency, voice quality, dimensions, rhythmic, fluency, spectral
  9. [9] § Methodology › Overview of the SpeechDETECT architecture › Component 2–voice analysis framework: characterizing vocal traits and temporal dynamics ↔ speechdetect/AcousticFeatureExtractor.py, lines 1419–1507 · score 0.56 · broader categories, speech fluency, map, voice
  10. [10] § Methodology › Overview of the SpeechDETECT architecture › Component 2.5. voice signal complexity ↔ speechdetect/acoustic_features/complexity.py, lines 127–145 · score 0.55 · Multiscale Permutation Entropy, MPE
  11. [11] § Methodology › Overview of the SpeechDETECT architecture › Component 2.5. voice signal complexity ↔ speechdetect/AcousticFeatureExtractor.py, lines 1185–1310 · score 0.54 · Fractal Dimension, Higuchi, HFD, modulate, amplitude, signal
  12. [12] § Methodology › Overview of the SpeechDETECT architecture › Component 2.4. loudness and intensity ↔ speechdetect/acoustic_features/loudness_and_intensity.py, lines 104–134 · score 0.53 · Sound Pressure, SPL, RMS, loudness, Intensity, Amplitude
  13. [13] § Methodology › Overview of the SpeechDETECT architecture › Component 3- computing acoustic and temporal features ↔ speechdetect/AcousticFeatureExtractor.py, lines 136–214 · score 0.51 · Cepstral Coefficients, Spectral Features, spectrogram, ms
  14. [14] § Results › SHAP values visualization (on Pitt corpus) ↔ speechdetect/AcousticFeatureExtractor.py, lines 355–395 · score 0.51 · pause ratio, pause duration, articulatory, syllable, Error, speech

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 1,507 lines · 69 KB · no license · 7 matches

  1. """
  2. AcousticFeatureExtractor - Main interface for extracting acoustic features from speech recordings
  3. """
  4. import torch
  5. import os
  6. import numpy as np
  7. import inspect
  8. import librosa
  9. import logging
  10. import csv
  11. from typing import List, Dict, Union, Any, Optional
  12. from .acoustic_features import (\
  14. get_pitch, calculate_time_varying_jitter, get_formants_frame_based,
  15. compute_msc, spectral_centriod, ltas, alpha_ratio, log_mel_spectrogram,
  16. mfcc, lpc, lpcc, spectral_envelope, calculate_cpp, hammIndex,
  17. plp_features, harmonicity, calculate_lsp_freqs_for_frames, calculate_frame_wise_zcr,
  18. analyze_audio_shimmer, calculate_frame_level_hnr, amplitude_range,
  19. rms_amplitude, spl_per_frame, peak_amplitude, short_time_energy, intensity,
  20. calculate_hfd_per_frame, calculate_frequency_entropy, calculate_amplitude_entropy,
  22. # Classes needed for process_file_model
  23. PauseBehavior, SpeechBehavior,
  25. # Statistical functions and names
  26. sma, de, function_names,
  27. )
  29. class AcousticFeatureExtractor:
  30. """
  31. A unified interface for extracting various acoustic features from speech recordings.
  32. """
  34. def __init__(self, sampling_rate=16000, log_level=logging.INFO):
  35. """
  36. Initialize the feature extractor
  38. Parameters:
  39. -----------
  40. sampling_rate : int
  41. Sampling rate of the audio files to process
  42. log_level : int
  43. Logging level (default: logging.INFO)
  44. """
  45. self.sampling_rate = sampling_rate
  47. # Configure logging
  48. self.logger = logging.getLogger(__name__)
  49. if not self.logger.handlers:
  50. handler = logging.StreamHandler()
  51. formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
  52. handler.setFormatter(formatter)
  53. self.logger.addHandler(handler)
  54. self.logger.setLevel(log_level)
  56. # Available feature types mapped to their extraction methods
  57. self.available_features = {
  58. 'spectral': self.extract_spectral_features,
  59. 'complexity': self.extract_complexity_features,
  60. 'frequency': self.extract_frequency_features,
  61. 'intensity': self.extract_intensity_features,
  62. 'rhythmic': self.extract_rhythmic_features,
  63. 'fluency': self.extract_fluency_features,
  64. 'voice_quality': self.extract_voice_quality_features,
  65. 'all': self.extract_all_features,
  66. 'raw': self.process_file,
  67. 'transcription': self.process_file_model
  68. }
  70. # VAD model and transcription model (to be set later if needed)
  71. self.vad_model = None
  72. self.vad_utils = None
  73. self.transcription_model = None
  75. self.logger.info("AcousticFeatureExtractor initialized with sampling rate %d Hz", sampling_rate)
  77. def set_models(self, vad_model=None, vad_utils=None, transcription_model=None):
  78. """
  79. Set the VAD and transcription models for advanced feature extraction
  81. Parameters:
  82. -----------
  83. vad_model : object
  84. Voice Activity Detection model
  85. vad_utils : object
  86. Utility functions for the VAD model
  87. transcription_model : object
  88. Transcription model for speech-to-text
  89. """
  90. self.vad_model = vad_model
  91. self.vad_utils = vad_utils
  92. self.transcription_model = transcription_model
  94. models_set = []
  95. if vad_model is not None:
  96. models_set.append("VAD model")
  97. if vad_utils is not None:
  98. models_set.append("VAD utils")
  99. if transcription_model is not None:
  100. models_set.append("transcription model")
  102. self.logger.info("Models configured: %s", ", ".join(models_set))
  104. def extract_all_features(self, audio_path):
  105. """
  106. Extract all available acoustic features from an audio file
  108. Parameters:
  109. -----------
  110. audio_path : str
  111. Path to the audio file
  113. Returns:
  114. --------
  115. dict
  116. Dictionary containing all extracted features
  117. """
  118. self.logger.info("Extracting all features from: %s", audio_path)
  119. features = {}
  121. # Extract all available feature sets except 'all' to avoid infinite recursion
  122. for feature_type, feature_method in self.available_features.items():
  123. # Skip 'all' to prevent infinite recursion
  124. if feature_type == 'all':
  125. continue
  127. try:
  128. feature_result = feature_method(audio_path)
  129. features.update(feature_result)
  130. except Exception as e:
  131. self.logger.error("Error extracting %s features: %s", feature_type, str(e))
  133. self.logger.info("Extracted %d features in total from 'all' option", len(features))
  134. return features
  136. def extract_spectral_features(self, audio_path):
  137. """Extract spectral features from audio"""
  138. self.logger.debug("Extracting spectral features from: %s", audio_path)
  139. try:
  140. # Load audio file
  141. data, fs = librosa.load(audio_path, sr=self.sampling_rate)
  143. # Define window parameters
  144. window_length_ms = 50
  145. window_step_ms = 25
  147. features = {}
  149. # Spectral features extraction
  150. try:
  151. # Modulation Spectrum Coefficients
  152. msc = compute_msc(data, fs, nfft=512, window_length_ms=window_length_ms, window_step_ms=window_step_ms, num_msc=13)
  153. features.update(self.process_matrix(msc, 'MSC'))
  155. # Spectral centroids
  156. centroids = spectral_centriod(data, fs, window_length_ms, window_step_ms)
  157. features.update(self.process_row(np.array(centroids), 'CENTRIODS'))
  159. # Long Term Average Spectrum
  160. LTAS, _ = ltas(data, fs, window_length_ms, window_step_ms)
  161. features.update(self.process_row(LTAS, 'LTAS'))
  163. # Alpha ratio
  164. ALPHA_RATIO = alpha_ratio(data, fs, window_length_ms, window_step_ms, (0, 1000), (1000, 5000))
  165. features['ALPHA_RATIO'] = ALPHA_RATIO
  167. # Log Mel Spectrogram
  168. LOG_MEL_SPECTROGRAM = log_mel_spectrogram(data, fs, window_length_ms, window_step_ms, melbands=8, fmin=20, fmax=6500)
  169. features.update(self.process_matrix(LOG_MEL_SPECTROGRAM, 'LOG_MEL_SPECTROGRAM'))
  171. # MFCCs
  172. MFCC = mfcc(data, fs, window_length_ms, window_step_ms, melbands=26, lifter=20)[:15]
  173. features.update(self.process_matrix(MFCC, 'MFCC'))
  175. # Linear Prediction Coefficients
  176. LPC = lpc(data, fs, window_length_ms, window_step_ms)
  177. features.update(self.process_matrix(LPC, 'LPC'))
  179. # Linear Prediction Cepstral Coefficients
  180. LPCC = lpcc(data, fs, window_length_ms, window_step_ms, lpc_length=8)
  181. features.update(self.process_matrix(LPCC, 'LPCC'))
  183. # Spectral Envelope
  184. ENVELOPE = spectral_envelope(data, fs, window_length_ms, window_step_ms)
  185. features.update(self.process_matrix(ENVELOPE, 'ENVELOPE'))
  187. # Cepstral Peak Prominence
  188. CPP = calculate_cpp(data, fs, window_length_ms, window_step_ms)
  189. features.update(self.process_row(CPP, 'CPP'))
  191. # Hammarberg Index
  192. HAMM_INDEX, _, _ = hammIndex(data, fs)
  193. features.update(self.process_row(HAMM_INDEX, 'HAMMARBERG_INDEX'))
  195. # Perceptual Linear Prediction
  196. PLP = plp_features(data, fs, num_filters=26, fmin=20, fmax=8000)
  197. features.update(self.process_matrix(PLP, 'PLP'))
  199. # Line Spectral Pairs
  200. lspFreq = calculate_lsp_freqs_for_frames(data, fs, window_length_ms, window_step_ms, order=8)
  201. features.update(self.process_matrix(lspFreq, "LSP"))
  203. # Zero Crossing Rate
  204. ZCR = calculate_frame_wise_zcr(data, fs, window_length_ms, window_step_ms)
  205. features.update(self.process_row(ZCR, "ZCR"))
  207. self.logger.info("Extracted %d spectral features", len(features))
  208. except Exception as e:
  209. self.logger.error(f"Error processing spectral features: {e}")
  211. return features
  212. except Exception as e:
  213. self.logger.error("Runtime error extracting spectral features: %s", str(e))
  214. return {}
  216. def extract_complexity_features(self, audio_path):
  217. """Extract complexity features from audio"""
  218. self.logger.debug("Extracting complexity features from: %s", audio_path)
  219. try:
  220. # Load audio file
  221. data, fs = librosa.load(audio_path, sr=self.sampling_rate)
  223. # Define window parameters
  224. window_length_ms = 50
  225. window_step_ms = 25
  227. features = {}
  229. # Complexity features extraction
  230. try:
  231. # Higuchi Fractal Dimension
  232. HFD = calculate_hfd_per_frame(data, fs, window_length_ms, window_step_ms, 10)
  233. features.update(self.process_row(HFD, 'HFD'))
  235. # Frequency Entropy
  236. FREQ_ENTROPY = calculate_frequency_entropy(data, fs, window_length_ms, window_step_ms)
  237. features.update(self.process_row(FREQ_ENTROPY, 'FREQ_ENTROPY'))
  239. # Amplitude Entropy
  240. AMP_ENTROPY = calculate_amplitude_entropy(data, fs, window_length_ms, window_step_ms)
  241. features.update(self.process_row(AMP_ENTROPY, 'AMP_ENTROPY'))
  243. # Zero Crossing Rate can also be considered a complexity feature
  244. ZCR = calculate_frame_wise_zcr(data, fs, window_length_ms, window_step_ms)
  245. features.update(self.process_row(ZCR, "ZCR"))
  247. self.logger.info("Extracted %d complexity features", len(features))
  248. except Exception as e:
  249. self.logger.error(f"Error processing complexity features: {e}")
  251. return features
  252. except Exception as e:
  253. self.logger.error("Runtime error extracting complexity features: %s", str(e))
  254. return {}
  256. def extract_frequency_features(self, audio_path):
  257. """Extract frequency features from audio"""
  258. self.logger.debug("Extracting frequency features from: %s", audio_path)
  259. try:
  260. # Load audio file
  261. data, fs = librosa.load(audio_path, sr=self.sampling_rate)
  263. # Define window parameters
  264. window_length_ms = 50
  265. window_step_ms = 25
  267. features = {}
  269. # Frequency parameters extraction
  270. try:
  271. # Fundamental Frequency (F0)
  272. F0 = get_pitch(data, fs, window_length_ms, window_step_ms)
  273. F0_valid = F0[~np.isnan(F0)]
  274. features.update(self.process_row(F0_valid, 'F0'))
  276. # Jitter
  277. jitter = calculate_time_varying_jitter(F0, fs, window_length_ms, window_step_ms, window_length_ms*2, window_step_ms*2)
  278. features.update(self.process_row(np.array(jitter), 'Jitter'))
  280. # Formants (F1, F2, F3)
  281. F_formants = get_formants_frame_based(data, fs, window_length_ms, window_step_ms, [1, 2, 3])
  282. # Handle the case where F_formants is returned as a tuple
  283. if isinstance(F_formants, tuple):
  284. # Extract the formant data from the tuple (first element is usually the formant data)
  285. formant_data = np.array(F_formants[0])
  286. else:
  287. formant_data = F_formants
  289. # Process each formant
  290. for i in range(formant_data.shape[0]):
  291. features.update(self.process_row(formant_data[i, :], f'F{i+1}'))
  293. # Harmonicity can also be considered a frequency feature
  294. HARMONICITY = harmonicity(data, fs)
  295. features['HARMONICITY'] = HARMONICITY
  297. self.logger.info("Extracted %d frequency features", len(features))
  298. except Exception as e:
  299. self.logger.error(f"Error processing frequency features: {e}")
  301. return features
  302. except Exception as e:
  303. self.logger.error("Runtime error extracting frequency features: %s", str(e))
  304. return {}
  306. def extract_intensity_features(self, audio_path):
  307. """Extract intensity and loudness features from audio"""
  308. self.logger.debug("Extracting intensity features from: %s", audio_path)
  309. try:
  310. # Load audio file
  311. data, fs = librosa.load(audio_path, sr=self.sampling_rate)
  313. # Define window parameters
  314. window_length_ms = 50
  315. window_step_ms = 25
  317. features = {}
  319. # Loudness and intensity parameters extraction
  320. try:
  321. # Root Mean Square amplitude
  322. RMS = rms_amplitude(data, fs, window_length_ms, window_step_ms)
  323. features.update(self.process_row(RMS, 'RMS'))
  325. # Sound Pressure Level
  326. SPL = spl_per_frame(data, fs, window_length_ms, window_step_ms)
  327. features.update(self.process_row(SPL, 'SPL'))
  329. # Peak amplitude
  330. PEAK = peak_amplitude(data, fs, window_length_ms, window_step_ms)
  331. features.update(self.process_row(PEAK, 'PEAK'))
  333. # Short-time energy
  334. STE = short_time_energy(data, fs, window_length_ms, window_step_ms)
  335. features.update(self.process_row(STE, 'STE'))
  337. # Intensity
  338. INTENSITY = intensity(data, fs, window_length_ms, window_step_ms)
  339. features.update(self.process_row(INTENSITY, 'INTENSITY'))
  341. # Amplitude range
  342. APQ_range, APQ_std = amplitude_range(data, fs, window_length_ms, window_step_ms)
  343. features.update(self.process_row(APQ_range, 'Amplitude_Range'))
  344. features.update(self.process_row(APQ_std, 'APQ2'))
  346. self.logger.info("Extracted %d intensity features", len(features))
  347. except Exception as e:
  348. self.logger.error(f"Error processing intensity features: {e}")
  350. return features
  351. except Exception as e:
  352. self.logger.error("Runtime error extracting intensity features: %s", str(e))
  353. return {}
  355. def extract_rhythmic_features(self, audio_path):
  356. """Extract rhythmic features from audio"""
  357. self.logger.debug("Extracting rhythmic features from: %s", audio_path)
  359. # Check if required models are available
  360. if None in (self.vad_model, self.vad_utils, self.transcription_model):
  361. self.logger.warning("VAD and transcription models are required for rhythmic features but not set")
  362. return {}
  364. try:
  365. features = {}
  367. # Create PauseBehavior instance for rhythmic analysis
  368. try:
  369. pause_behavior = PauseBehavior(self.vad_model, self.vad_utils, self.transcription_model)
  370. pause_behavior.configure(audio_path)
  372. # List of methods to extract from pause_behavior for rhythmic features
  373. rhythmic_methods = [
  374. 'syllable_rate', 'speech_rate', 'articulation_rate',
  375. 'mean_pause_duration', 'mean_silence_duration', 'mean_speech_duration',
  376. 'speech_to_pause_ratio', 'percentage_silence', 'percentage_voice'
  377. ]
  379. # Extract relevant rhythmic features
  380. for name in rhythmic_methods:
  381. method = getattr(pause_behavior, name, None)
  382. if method and callable(method):
  383. try:
  384. features[name] = method()
  385. except Exception as e:
  386. self.logger.error(f"Error executing PauseBehavior method {name}: {e}")
  388. self.logger.info("Extracted %d rhythmic features", len(features))
  389. except Exception as e:
  390. self.logger.error(f"Error in rhythmic feature extraction: {e}")
  392. return features
  393. except Exception as e:
  394. self.logger.error("Runtime error extracting rhythmic features: %s", str(e))
  395. return {}
  397. def extract_fluency_features(self, audio_path):
  398. """Extract speech fluency features from audio"""
  399. self.logger.debug("Extracting fluency features from: %s", audio_path)
  401. # Check if required models are available
  402. if None in (self.vad_model, self.vad_utils, self.transcription_model):
  403. self.logger.warning("VAD and transcription models are required for fluency features but not set")
  404. return {}
  406. try:
  407. features = {}
  409. # Create SpeechBehavior instance for fluency analysis
  410. try:
  411. # First, set up PauseBehavior to get speech segmentation
  412. pause_behavior = PauseBehavior(self.vad_model, self.vad_utils, self.transcription_model)
  413. pause_behavior.configure(audio_path)
  415. # Then, set up SpeechBehavior using data from PauseBehavior
  416. speech_behavior = SpeechBehavior(self.vad_model, self.vad_utils, self.transcription_model)
  418. # Copy data from pause_behavior to speech_behavior
  419. speech_behavior.data = pause_behavior.data
  420. speech_behavior.silence_ranges = pause_behavior.silence_ranges
  421. speech_behavior.speech_ranges = pause_behavior.speech_ranges
  422. speech_behavior.transcription_result = pause_behavior.transcription_result
  423. speech_behavior.text = pause_behavior.text
  425. # Perform phoneme alignment
  426. speech_behavior.phoneme_alignment(audio_path)
  428. # List of methods to extract from speech_behavior for fluency features
  429. fluency_methods = [
  430. 'phonation_rate', 'phonation_time', 'articulation_time',
  431. 'mean_duration_of_bursts', 'number_of_pauses', 'number_of_filled_pauses',
  432. 'filled_pauses_per_min', 'mean_length_of_runs', 'hesitation_ratio'
  433. ]
  435. # Extract relevant fluency features
  436. for name in fluency_methods:
  437. method = getattr(speech_behavior, name, None)
  438. if method and callable(method):
  439. try:
  440. features[name] = method()
  441. except Exception as e:
  442. self.logger.error(f"Error executing SpeechBehavior method {name}: {e}")
  444. # Process regularity and PVI features
  445. try:
  446. for i, res in enumerate(speech_behavior.regularity_of_segments()):
  447. features[f"regularity_{i}"] = res
  448. for i, res in enumerate(speech_behavior.alternating_regularity()):
  449. features[f"PVI_{i}"] = res
  450. except Exception as e_reg:
  451. self.logger.error(f"Error processing regularity/PVI features: {e_reg}")
  453. # Process relative sentence duration
  454. try:
  455. relative_sentence_duration = speech_behavior.relative_sentence_duration()
  456. features.update(self.process_row(np.array(relative_sentence_duration), "relative_sentence_duration"))
  457. except Exception as e_rel:
  458. self.logger.error(f"Error processing relative sentence duration: {e_rel}")
  460. self.logger.info("Extracted %d fluency features", len(features))
  461. except Exception as e:
  462. self.logger.error(f"Error in fluency feature extraction: {e}")
  464. return features
  465. except Exception as e:
  466. self.logger.error("Runtime error extracting fluency features: %s", str(e))
  467. return {}
  469. def extract_voice_quality_features(self, audio_path):
  470. """Extract voice quality features from audio"""
  471. self.logger.debug("Extracting voice quality features from: %s", audio_path)
  472. try:
  473. # Load audio file
  474. data, fs = librosa.load(audio_path, sr=self.sampling_rate)
  476. # Define window parameters
  477. window_length_ms = 50
  478. window_step_ms = 25
  480. features = {}
  482. # Voice quality features extraction
  483. try:
  484. # Shimmer
  485. SHIMMER = analyze_audio_shimmer(data, fs, window_length_ms, window_step_ms)
  486. features.update(self.process_row(SHIMMER, 'SHIMMER'))
  488. # Harmonics-to-Noise Ratio and Noise-to-Harmonics Ratio
  489. HNR, NHR = calculate_frame_level_hnr(data, fs, window_length_ms, window_step_ms)
  490. features.update(self.process_row(HNR, 'HNR'))
  491. features.update(self.process_row(NHR, 'NHR'))
  493. # Harmonicity
  494. HARMONICITY = harmonicity(data, fs)
  495. features['HARMONICITY'] = HARMONICITY
  497. # Cepstral Peak Prominence (also a voice quality feature)
  498. CPP = calculate_cpp(data, fs, window_length_ms, window_step_ms)
  499. features.update(self.process_row(CPP, 'CPP'))
  501. # Hammarberg Index (spectral tilt related to voice quality)
  502. HAMM_INDEX, _, _ = hammIndex(data, fs)
  503. features.update(self.process_row(HAMM_INDEX, 'HAMMARBERG_INDEX'))
  505. # Alpha Ratio (spectral tilt related to voice quality)
  506. ALPHA_RATIO = alpha_ratio(data, fs, window_length_ms, window_step_ms, (0, 1000), (1000, 5000))
  507. features['ALPHA_RATIO'] = ALPHA_RATIO
  509. # Jitter (also a voice quality feature)
  510. F0 = get_pitch(data, fs, window_length_ms, window_step_ms)
  511. jitter = calculate_time_varying_jitter(F0, fs, window_length_ms, window_step_ms, window_length_ms*2, window_step_ms*2)
  512. features.update(self.process_row(np.array(jitter), 'Jitter'))
  514. self.logger.info("Extracted %d voice quality features", len(features))
  515. except Exception as e:
  516. self.logger.error(f"Error processing voice quality features: {e}")
  518. return features
  519. except Exception as e:
  520. self.logger.error("Runtime error extracting voice quality features: %s", str(e))
  521. return {}
  523. def process_row(self, row, feature_name, index=-1):
  524. """
  525. Process a single row of data with statistical functions
  527. Parameters:
  528. -----------
  529. row : numpy.ndarray
  530. Row of data to process
  531. feature_name : str
  532. Name of the feature
  533. index : int, optional
  534. Index for matrix features, -1 for non-matrix features
  536. Returns:
  537. --------
  538. dict
  539. Dictionary of processed features
  540. """
  541. # Import all statistical functions individually
  542. from .acoustic_features.statistical_functions import (
  543. max, min, span, maxPos, minPos, amean, linregc1, linregc2,
  544. linregerrA, linregerrQ, stddev, skewness, kurtosis,
  545. quartile1, quartile2, quartile3, iqr1_2, iqr2_3, iqr1_3,
  546. percentile1, percentile99, pctlrange0_1, upleveltime75, upleveltime90
  547. )
  549. # Create a dictionary mapping function names to the actual function
  550. stat_funcs = {
  551. "max": max, "min": min, "span": span, "maxPos": maxPos, "minPos": minPos,
  552. "amean": amean, "linregc1": linregc1, "linregc2": linregc2,
  553. "linregerrA": linregerrA, "linregerrQ": linregerrQ, "stddev": stddev,
  554. "skewness": skewness, "kurtosis": kurtosis, "quartile1": quartile1,
  555. "quartile2": quartile2, "quartile3": quartile3, "iqr1_2": iqr1_2,
  556. "iqr2_3": iqr2_3, "iqr1_3": iqr1_3, "percentile1": percentile1,
  557. "percentile99": percentile99, "pctlrange0_1": pctlrange0_1,
  558. "upleveltime75": upleveltime75, "upleveltime90": upleveltime90
  559. }
  561. row_sma = sma(row)
  562. results = {}
  564. # Apply statistical functions to smoothed signal
  565. for func_name in function_names:
  566. # Get the function from our mapping instead of globals()
  567. func = stat_funcs.get(func_name)
  568. if func is None:
  569. self.logger.warning(f"Statistical function {func_name} not found in stat_funcs mapping.")
  570. continue
  571. try:
  572. result = func(row_sma)
  573. except Exception as e:
  574. self.logger.error(f"Error applying {func_name} to {feature_name}: {e}")
  575. result = None
  577. if index >= 0:
  578. name = f"{feature_name}_sma[{index}]_{func_name}"
  579. else:
  580. name = f"{feature_name}_sma_{func_name}"
  582. results[name] = result
  584. # Apply statistical functions to derivative of smoothed signal
  585. row_sma_de = de(row_sma)
  586. for func_name in function_names:
  587. func = stat_funcs.get(func_name)
  588. if func is None:
  589. self.logger.warning(f"Statistical function {func_name} not found in stat_funcs mapping.")
  590. continue
  591. try:
  592. result = func(row_sma_de)
  593. except Exception as e:
  594. self.logger.error(f"Error applying {func_name} to derivative of {feature_name}: {e}")
  595. result = None
  597. if index >= 0:
  598. name = f"{feature_name}_sma_de[{index}]_{func_name}"
  599. else:
  600. name = f"{feature_name}_sma_de_{func_name}"
  602. results[name] = result
  604. return results
  606. def process_matrix(self, matrix, feature_name):
  607. """
  608. Process a matrix of features with statistical functions
  610. Parameters:
  611. -----------
  612. matrix : numpy.ndarray
  613. Matrix to process
  614. feature_name : str
  615. Name of the feature
  617. Returns:
  618. --------
  619. dict
  620. Dictionary of processed features
  621. """
  622. matrix_results = {}
  623. for i, row in enumerate(matrix):
  624. result = self.process_row(row, feature_name, i)
  625. matrix_results.update(result)
  627. return matrix_results
  629. def length(self, res):
  630. """
  631. Count the total number of features in a list of dictionaries
  633. Parameters:
  634. -----------
  635. res : list
  636. List of dictionaries
  638. Returns:
  639. --------
  640. int
  641. Total number of features
  642. """
  643. return sum([len(elm) for elm in res])
  645. def process_file(self, filepath):
  646. """
  647. Process a file and extract all raw acoustic features
  649. Parameters:
  650. -----------
  651. filepath : str
  652. Path to the audio file
  654. Returns:
  655. --------
  656. dict
  657. Dictionary of all extracted features
  658. """
  659. self.logger.info("Processing file for raw feature extraction: %s", filepath)
  661. try:
  662. data, fs = librosa.load(filepath, sr=self.sampling_rate)
  663. except Exception as e:
  664. self.logger.error(f"Failed to load audio file {filepath}: {e}")
  665. return {}
  667. # Define window parameters
  668. window_length_ms = 50
  669. window_step_ms = 25
  671. # --- Feature Calculation with Error Handling ---
  672. features = {}
  674. # Frequency parameters
  675. try:
  676. F0 = get_pitch(data, fs, window_length_ms, window_step_ms)
  677. F0_valid = F0[~np.isnan(F0)]
  678. features.update(self.process_row(F0_valid, 'F0'))
  680. jitter = calculate_time_varying_jitter(F0, fs, window_length_ms, window_step_ms, window_length_ms*2, window_step_ms*2)
  681. features.update(self.process_row(np.array(jitter), 'Jitter'))
  683. F_formants = get_formants_frame_based(data, fs, window_length_ms, window_step_ms, [1, 2, 3])
  684. # Handle the case where F_formants is returned as a tuple
  685. if isinstance(F_formants, tuple):
  686. # Extract the formant data from the tuple (first element is usually the formant data)
  687. formant_data = np.array(F_formants[0])
  688. else:
  689. formant_data = F_formants
  691. # Process each formant
  692. for i in range(formant_data.shape[0]):
  693. features.update(self.process_row(formant_data[i, :], f'F{i+1}'))
  694. self.logger.info("Processed frequency parameters")
  695. except Exception as e:
  696. self.logger.error(f"Error processing frequency parameters: {e}")
  698. # Spectral features
  699. try:
  700. msc = compute_msc(data, fs, nfft=512, window_length_ms=window_length_ms, window_step_ms=window_step_ms, num_msc=13)
  701. features.update(self.process_matrix(msc, 'MSC'))
  703. centroids = spectral_centriod(data, fs, window_length_ms, window_step_ms)
  704. features.update(self.process_row(np.array(centroids), 'CENTRIODS'))
  706. LTAS, _ = ltas(data, fs, window_length_ms, window_step_ms) # Ignore freq array return
  707. features.update(self.process_row(LTAS, 'LTAS'))
  709. ALPHA_RATIO = alpha_ratio(data, fs, window_length_ms, window_step_ms, (0, 1000), (1000, 5000))
  710. # Alpha ratio is often scalar, handle differently if needed, here adding directly
  711. features['ALPHA_RATIO'] = ALPHA_RATIO
  713. LOG_MEL_SPECTROGRAM = log_mel_spectrogram(data, fs, window_length_ms, window_step_ms, melbands=8, fmin=20, fmax=6500)
  714. features.update(self.process_matrix(LOG_MEL_SPECTROGRAM, 'LOG_MEL_SPECTROGRAM'))
  716. MFCC = mfcc(data, fs, window_length_ms, window_step_ms, melbands=26, lifter=20)[:15]
  717. features.update(self.process_matrix(MFCC, 'MFCC'))
  719. LPC = lpc(data, fs, window_length_ms, window_step_ms)
  720. features.update(self.process_matrix(LPC, 'LPC'))
  722. LPCC = lpcc(data, fs, window_length_ms, window_step_ms, lpc_length=8)
  723. features.update(self.process_matrix(LPCC, 'LPCC'))
  725. ENVELOPE = spectral_envelope(data, fs, window_length_ms, window_step_ms)
  726. features.update(self.process_matrix(ENVELOPE, 'ENVELOPE'))
  728. CPP = calculate_cpp(data, fs, window_length_ms, window_step_ms)
  729. features.update(self.process_row(CPP, 'CPP'))
  731. HAMM_INDEX, _, _ = hammIndex(data, fs) # Ignore freq and Pxx return
  732. features.update(self.process_row(HAMM_INDEX, 'HAMMARBERG_INDEX'))
  734. PLP = plp_features(data, fs, num_filters=26, fmin=20, fmax=8000)
  735. features.update(self.process_matrix(PLP, 'PLP'))
  737. HARMONICITY = harmonicity(data, fs)
  738. features['HARMONICITY'] = HARMONICITY # Often scalar
  740. lspFreq = calculate_lsp_freqs_for_frames(data, fs, window_length_ms, window_step_ms, order=8)
  741. features.update(self.process_matrix(lspFreq, "LSP"))
  743. ZCR = calculate_frame_wise_zcr(data, fs, window_length_ms, window_step_ms)
  744. features.update(self.process_row(ZCR, "ZCR"))
  745. self.logger.info("Processed spectral domain features")
  746. except Exception as e:
  747. self.logger.error(f"Error processing spectral features: {e}")
  749. # Voice quality
  750. try:
  751. SHIMMER = analyze_audio_shimmer(data, fs, window_length_ms, window_step_ms)
  752. features.update(self.process_row(SHIMMER, 'SHIMMER'))
  754. HNR, NHR = calculate_frame_level_hnr(data, fs, window_length_ms, window_step_ms)
  755. features.update(self.process_row(HNR, 'HNR'))
  756. features.update(self.process_row(NHR, 'NHR'))
  758. # Amplitude range returns range and std deviation
  759. APQ_range, APQ_std = amplitude_range(data, fs, window_length_ms, window_step_ms)
  760. features.update(self.process_row(APQ_range, 'Amplitude_Range')) # Renamed from APQ_range
  761. features.update(self.process_row(APQ_std, 'APQ2')) # Renamed from APQ_std
  762. self.logger.info("Processed voice quality features")
  763. except Exception as e:
  764. self.logger.error(f"Error processing voice quality features: {e}")
  766. # Loudness and intensity
  767. try:
  768. RMS = rms_amplitude(data, fs, window_length_ms, window_step_ms)
  769. features.update(self.process_row(RMS, 'RMS'))
  771. SPL = spl_per_frame(data, fs, window_length_ms, window_step_ms)
  772. features.update(self.process_row(SPL, 'SPL'))
  774. PEAK = peak_amplitude(data, fs, window_length_ms, window_step_ms)
  775. features.update(self.process_row(PEAK, 'PEAK'))
  777. STE = short_time_energy(data, fs, window_length_ms, window_step_ms)
  778. features.update(self.process_row(STE, 'STE'))
  780. INTENSITY = intensity(data, fs, window_length_ms, window_step_ms)
  781. features.update(self.process_row(INTENSITY, 'INTENSITY'))
  782. self.logger.info("Processed loudness and intensity parameters")
  783. except Exception as e:
  784. self.logger.error(f"Error processing loudness and intensity features: {e}")
  786. # Complexity
  787. try:
  788. HFD = calculate_hfd_per_frame(data, fs, window_length_ms, window_step_ms, 10)
  789. features.update(self.process_row(HFD, 'HFD'))
  791. FREQ_ENTROPY = calculate_frequency_entropy(data, fs, window_length_ms, window_step_ms)
  792. features.update(self.process_row(FREQ_ENTROPY, 'FREQ_ENTROPY'))
  794. AMP_ENTROPY = calculate_amplitude_entropy(data, fs, window_length_ms, window_step_ms)
  795. features.update(self.process_row(AMP_ENTROPY, 'AMP_ENTROPY'))
  796. self.logger.info("Processed complexity features")
  797. except Exception as e:
  798. self.logger.error(f"Error processing complexity features: {e}")
  800. self.logger.info("Extracted a total of %d raw features", len(features))
  801. return features
  803. def process_file_model(self, filepath):
  804. """
  805. Process a file using VAD and transcription models
  807. Parameters:
  808. -----------
  809. filepath : str
  810. Path to the audio file
  812. Returns:
  813. --------
  814. dict
  815. Dictionary of all extracted features
  817. Raises:
  818. -------
  819. ValueError
  820. If models are not set
  821. ImportError
  822. If required dependencies for fluency/rhythmic features are missing
  823. """
  824. if None in (self.vad_model, self.vad_utils, self.transcription_model):
  825. raise ValueError("VAD and transcription models must be set using set_models() before calling this method")
  826. # Raise the specific import error if possible, otherwise a generic one
  827. try:
  828. # Attempting to instantiate will raise the specific error if placeholder is used
  829. PauseBehavior(None, None, None)
  830. SpeechBehavior(None, None, None)
  831. except Exception:
  832. raise ImportError("Required dependencies for fluency/rhythmic features are missing.")
  834. self.logger.info("Processing file with VAD and transcription models: %s", filepath)
  836. p_results = {}
  837. s_results = {}
  838. prob = {}
  840. try:
  841. # Process rhythmic features
  842. pause_behavior = PauseBehavior(self.vad_model, self.vad_utils, self.transcription_model)
  843. pause_behavior.configure(filepath)
  845. voiceProb_signal = self.vad_model.audio_forward(pause_behavior.data, sr=16000)
  846. voiceProb_signal = np.array(voiceProb_signal[0])
  847. prob = self.process_row(voiceProb_signal, "voiceProb")
  849. # List of methods to exclude
  850. excluded_methods_p = ['__init__', 'configure']
  852. # Iterate through all methods of the pause_behavior instance
  853. self.logger.debug("Extracting pause behavior features")
  854. for name, method in inspect.getmembers(pause_behavior, predicate=inspect.ismethod):
  855. if name not in excluded_methods_p:
  856. try:
  857. p_results[name] = method()
  858. except Exception as e_method:
  859. self.logger.error(f"Error executing PauseBehavior method {name}: {e_method}")
  861. # Process speech features
  862. speech_behavior = SpeechBehavior(self.vad_model, self.vad_utils, self.transcription_model)
  864. # Copy data from pause_behavior to speech_behavior
  865. speech_behavior.data = pause_behavior.data
  866. speech_behavior.silence_ranges = pause_behavior.silence_ranges
  867. speech_behavior.speech_ranges = pause_behavior.speech_ranges
  868. speech_behavior.transcription_result = pause_behavior.transcription_result
  869. speech_behavior.text = pause_behavior.text
  871. # Perform phoneme alignment
  872. alignment_error = speech_behavior.phoneme_alignment(filepath)
  873. self.logger.info("Phoneme alignment completed (total segments: %d, hypothetical: %d)",
  874. alignment_error[0], alignment_error[1])
  876. # List of methods to exclude
  877. excluded_methods_s = ['__init__', 'configure', 'phoneme_alignment', 'relative_sentence_duration', 'regularity_of_segments', 'alternating_regularity']
  879. # Iterate through all methods of the speech_behavior instance
  880. self.logger.debug("Extracting speech behavior features")
  881. for name, method in inspect.getmembers(speech_behavior, predicate=inspect.ismethod):
  882. if name not in excluded_methods_s:
  883. try:
  884. s_results[name] = method()
  885. except Exception as e_method:
  886. self.logger.error(f"Error executing SpeechBehavior method {name}: {e_method}")
  888. # Process regularity and PVI features
  889. self.logger.debug("Processing regularity and PVI features")
  890. try:
  891. for i, res in enumerate(speech_behavior.regularity_of_segments()):
  892. s_results[f"regularity_{i}"] = res
  893. for i, res in enumerate(speech_behavior.alternating_regularity()):
  894. s_results[f"PVI_{i}"] = res
  895. except Exception as e_reg:
  896. self.logger.error(f"Error processing regularity/PVI features: {e_reg}")
  898. # Process relative sentence duration
  899. try:
  900. relative_sentence_duration = speech_behavior.relative_sentence_duration()
  901. s_results.update(self.process_row(np.array(relative_sentence_duration), "relative_sentence_duration"))
  902. except Exception as e_rel:
  903. self.logger.error(f"Error processing relative sentence duration: {e_rel}")
  905. except Exception as e_main:
  906. self.logger.critical(f"Major error during model-based processing: {e_main}")
  907. # Depending on severity, you might want to return partial results or an empty dict
  908. # return {}
  910. # Merge all results
  911. final_results = {**p_results, **prob, **s_results}
  912. self.logger.debug("Feature counts - pause: %d, prob: %d, speech: %d", len(p_results), len(prob), len(s_results))
  914. self.logger.info("Extracted a total of %d features with VAD and transcription models", len(final_results))
  915. return final_results
  917. def extract_features(self, audio_paths: Union[str, List[str], 'torch.utils.data.DataLoader'], features_to_calculate: Optional[List[str]] = None, separate_groups: bool = False, output_dir: Optional[str] = None) -> Dict[str, Any]:
  918. """
  919. Extract specified features from a single audio file, a batch of files, or a PyTorch DataLoader
  921. Parameters:
  922. -----------
  923. audio_paths : str or List[str] or torch.utils.data.DataLoader
  924. Path(s) to the audio file(s) or a PyTorch DataLoader that yields:
  925. - File paths as strings
  926. - Tuples/lists where the first element is a file path
  927. - Audio tensors with shape (batch_size, num_samples) or (num_samples,)
  928. features_to_calculate : List[str], optional
  929. List of feature types to extract. If None, extracts all features.
  930. Valid options: 'spectral', 'complexity', 'frequency', 'intensity',
  931. 'rhythmic', 'fluency', 'voice_quality', 'all',
  932. 'raw', 'transcription'
  933. separate_groups : bool, optional
  934. If True, organizes output by feature groups. If False (default), returns a flat dictionary.
  935. output_dir : str, optional
  936. If provided, extracted features for each file will be saved as a CSV file in this directory
  937. with the same name as the audio file.
  939. Returns:
  940. --------
  941. dict
  942. Dictionary containing extracted features
  943. If separate_groups=False (default):
  944. For a single file: {feature_name: feature_value, ...}
  945. For multiple files/DataLoader: {file_path/idx: {feature_name: feature_value, ...}, ...}
  946. If separate_groups=True:
  947. For a single file: {feature_group: {feature_name: feature_value, ...}, ...}
  948. For multiple files/DataLoader: {file_path/idx: {feature_group: {feature_name: feature_value, ...}, ...}, ...}
  949. """
  950. # Default to all features if none specified
  951. if features_to_calculate is None:
  952. features_to_calculate = ['all']
  954. self.logger.info("Extracting features: %s", ", ".join(features_to_calculate))
  956. # Check if output directory exists and create it if needed
  957. if output_dir is not None:
  958. os.makedirs(output_dir, exist_ok=True)
  959. self.logger.info(f"Features will be saved to directory: {output_dir}")
  961. # Check if feature types are valid and available
  962. requested_unavailable = []
  963. for feature_type in features_to_calculate:
  964. if feature_type not in self.available_features:
  965. raise ValueError(f"Unknown feature type: {feature_type}. Available types: {list(self.available_features.keys())}")
  967. if requested_unavailable:
  968. self.logger.warning(
  969. "The following requested feature types are unavailable due to missing dependencies and will be skipped: %s",
  970. ", ".join(requested_unavailable)
  971. )
  973. # Check if input is a PyTorch DataLoader
  974. try:
  975. import torch
  976. from torch.utils.data import DataLoader
  977. is_dataloader = isinstance(audio_paths, DataLoader)
  978. except ImportError:
  979. is_dataloader = False
  981. # Process a PyTorch DataLoader
  982. if is_dataloader:
  983. self.logger.info("Processing PyTorch DataLoader")
  984. batch_results = {}
  986. # Process each batch from the DataLoader
  987. for batch_idx, batch in enumerate(audio_paths):
  988. self.logger.info(f"Processing batch {batch_idx+1}")
  990. # Handle different batch formats:
  991. # 1. Batch of file paths (strings)
  992. # 2. Batch of tuples where first element is file path
  993. # 3. Batch of audio tensors
  995. if isinstance(batch, torch.Tensor):
  996. # Batch is a tensor containing audio samples
  997. if len(batch.shape) == 1: # Single audio sample
  998. self.logger.debug("Processing single audio tensor")
  999. # Convert tensor to numpy array
  1000. audio_data = batch.cpu().numpy()
  1001. # Process single audio array
  1002. file_result = self._process_audio_data(
  1003. audio_data,
  1004. features_to_calculate,
  1005. f"tensor_batch{batch_idx}"
  1006. )
  1007. key = f"batch{batch_idx}_item0"
  1009. if separate_groups and file_result:
  1010. self.logger.info("Extracted %d features total for %s", len(file_result), key)
  1011. grouped_file_result = self._organize_features_by_group(file_result)
  1012. batch_results[key] = grouped_file_result
  1013. else:
  1014. batch_results[key] = file_result
  1016. # Save features to CSV if output_dir is provided
  1017. if output_dir is not None and file_result:
  1018. csv_path = os.path.join(output_dir, f"{key}.csv")
  1019. self._save_features_to_csv(file_result, csv_path)
  1021. else: # Batch of audio samples
  1022. self.logger.debug(f"Processing batch of {batch.shape[0]} audio tensors")
  1023. for i in range(batch.shape[0]):
  1024. audio_data = batch[i].cpu().numpy()
  1025. file_result = self._process_audio_data(
  1026. audio_data,
  1027. features_to_calculate,
  1028. f"tensor_batch{batch_idx}_item{i}"
  1029. )
  1030. key = f"batch{batch_idx}_item{i}"
  1032. if separate_groups and file_result:
  1033. self.logger.info("Extracted %d features total for %s", len(file_result), key)
  1034. grouped_file_result = self._organize_features_by_group(file_result)
  1035. batch_results[key] = grouped_file_result
  1036. else:
  1037. batch_results[key] = file_result
  1039. # Save features to CSV if output_dir is provided
  1040. if output_dir is not None and file_result:
  1041. csv_path = os.path.join(output_dir, f"{key}.csv")
  1042. self._save_features_to_csv(file_result, csv_path)
  1044. elif isinstance(batch, (list, tuple)):
  1045. # Batch is a list or tuple - check first element
  1046. if len(batch) > 0 and isinstance(batch[0], str):
  1047. # Batch contains file paths
  1048. for i, item in enumerate(batch):
  1049. file_result = self._process_single_file(item, features_to_calculate)
  1051. if separate_groups and file_result:
  1052. self.logger.info("Extracted %d features total for %s", len(file_result), item)
  1053. grouped_file_result = self._organize_features_by_group(file_result)
  1054. batch_results[item] = grouped_file_result
  1055. else:
  1056. batch_results[item] = file_result
  1058. # Save features to CSV if output_dir is provided
  1059. if output_dir is not None and file_result:
  1060. file_name = os.path.splitext(os.path.basename(item))[0]
  1061. csv_path = os.path.join(output_dir, f"{file_name}.csv")
  1062. self._save_features_to_csv(file_result, csv_path)
  1064. elif len(batch) > 0 and isinstance(batch[0], (list, tuple)) and len(batch[0]) > 0 and isinstance(batch[0][0], str):
  1065. # Batch contains tuples/lists where first element is a file path
  1066. for i, item in enumerate(batch):
  1067. file_path = item[0] # Get the file path (first element)
  1068. file_result = self._process_single_file(file_path, features_to_calculate)
  1070. if separate_groups and file_result:
  1071. self.logger.info("Extracted %d features total for %s", len(file_result), file_path)
  1072. grouped_file_result = self._organize_features_by_group(file_result)
  1073. batch_results[file_path] = grouped_file_result
  1074. else:
  1075. batch_results[file_path] = file_result
  1077. # Save features to CSV if output_dir is provided
  1078. if output_dir is not None and file_result:
  1079. file_name = os.path.splitext(os.path.basename(file_path))[0]
  1080. csv_path = os.path.join(output_dir, f"{file_name}.csv")
  1081. self._save_features_to_csv(file_result, csv_path)
  1083. else:
  1084. self.logger.warning(f"Unsupported batch format. Skipping batch {batch_idx}")
  1085. continue
  1087. else:
  1088. self.logger.warning(f"Unsupported batch item type: {type(batch)}. Skipping batch {batch_idx}")
  1089. continue
  1091. self.logger.info("DataLoader processing complete. Processed %d items total.", len(batch_results))
  1092. return batch_results
  1094. # Process a single file
  1095. elif isinstance(audio_paths, str):
  1096. result = self._process_single_file(audio_paths, features_to_calculate, separate_groups)
  1098. # Save features to CSV if output_dir is provided
  1099. if output_dir is not None and result:
  1100. file_name = os.path.splitext(os.path.basename(audio_paths))[0]
  1101. csv_path = os.path.join(output_dir, f"{file_name}.csv")
  1103. # For single file, the result might be already grouped
  1104. if separate_groups:
  1105. # Flatten the grouped features for CSV output
  1106. flattened_features = self._flatten_grouped_features(result)
  1107. self._save_features_to_csv(flattened_features, csv_path)
  1108. else:
  1109. self._save_features_to_csv(result, csv_path)
  1111. return result
  1113. # Process batch of files
  1114. else:
  1115. self.logger.info("Processing batch of %d files", len(audio_paths))
  1116. batch_results = {}
  1118. for i, audio_path in enumerate(audio_paths):
  1119. self.logger.info("Processing file %d/%d: %s", i+1, len(audio_paths), audio_path)
  1120. file_result = self._process_single_file(audio_path, features_to_calculate)
  1122. # If separate_groups is True, organize features by group for each file
  1123. if separate_groups and file_result:
  1124. self.logger.info("Extracted %d features total for %s", len(file_result), audio_path)
  1125. grouped_file_result = self._organize_features_by_group(file_result)
  1126. batch_results[audio_path] = grouped_file_result
  1127. else:
  1128. batch_results[audio_path] = file_result
  1130. # Save features to CSV if output_dir is provided
  1131. if output_dir is not None and file_result:
  1132. file_name = os.path.splitext(os.path.basename(audio_path))[0]
  1133. csv_path = os.path.join(output_dir, f"{file_name}.csv")
  1134. self._save_features_to_csv(file_result, csv_path)
  1136. self.logger.info("Batch processing complete")
  1137. return batch_results
  1139. def _process_single_file(self, audio_path, features_to_calculate, separate_groups=False):
  1140. """
  1141. Process a single audio file and extract features
  1143. Parameters:
  1144. -----------
  1145. audio_path : str
  1146. Path to the audio file
  1147. features_to_calculate : List[str]
  1148. List of feature types to extract
  1149. separate_groups : bool
  1150. If True, organizes output by feature groups
  1152. Returns:
  1153. --------
  1154. dict
  1155. Dictionary of extracted features
  1156. """
  1157. self.logger.info("Processing single file: %s", audio_path)
  1158. result = {}
  1160. for feature_type in features_to_calculate:
  1161. # If 'all' is included, extract all available features and skip other types
  1162. if feature_type == 'all':
  1163. result = self.extract_all_features(audio_path)
  1164. break
  1166. # Extract each requested feature type
  1167. # Add try-except around the call for robustness
  1168. try:
  1169. feature_extractor = self.available_features[feature_type]
  1170. features = feature_extractor(audio_path)
  1171. result.update(features)
  1172. except Exception as e:
  1173. self.logger.error(f"Error during extraction of '{feature_type}' features for {audio_path}: {e}")
  1175. # If separate_groups is True, organize features by group
  1176. if separate_groups and result:
  1177. self.logger.info("Extracted %d features total for %s", len(result), audio_path)
  1178. grouped_result = self._organize_features_by_group(result)
  1179. self.logger.info("Extracted features organized into %d groups for %s", len(grouped_result), audio_path)
  1180. return grouped_result
  1182. self.logger.info("Extracted %d features total for %s", len(result), audio_path)
  1183. return result
  1185. def _process_audio_data(self, audio_data, features_to_calculate, identifier="audio_tensor"):
  1186. """
  1187. Process audio data directly from a numpy array and extract features
  1189. Parameters:
  1190. -----------
  1191. audio_data : numpy.ndarray
  1192. Audio samples as a numpy array
  1193. features_to_calculate : List[str]
  1194. List of feature types to extract
  1195. identifier : str
  1196. Identifier for logging purposes
  1198. Returns:
  1199. --------
  1200. dict
  1201. Dictionary of extracted features
  1202. """
  1203. self.logger.info("Processing audio data: %s", identifier)
  1204. result = {}
  1206. # Create a function that processes audio data for each feature type
  1207. def process_audio_with_feature(feature_type):
  1208. # Skip unsupported feature types for direct audio data
  1209. if feature_type in ['rhythmic', 'fluency', 'transcription']:
  1210. self.logger.warning(f"Feature type '{feature_type}' not supported for direct audio data processing")
  1211. return {}
  1213. if feature_type == 'all':
  1214. # For 'all', just call each supported feature type
  1215. all_features = {}
  1216. for ft in ['spectral', 'complexity', 'frequency', 'intensity', 'voice_quality']:
  1217. all_features.update(process_audio_with_feature(ft))
  1218. return all_features
  1220. # Extract features directly from audio data
  1221. try:
  1222. # Most feature extraction functions require signal and sampling rate
  1223. fs = self.sampling_rate
  1224. data = audio_data
  1226. window_length_ms = 50
  1227. window_step_ms = 25
  1229. features = {}
  1231. if feature_type == 'spectral':
  1232. # Extract spectral features from data
  1233. try:
  1234. # Modulation Spectrum Coefficients
  1235. msc = compute_msc(data, fs, nfft=512, window_length_ms=window_length_ms, window_step_ms=window_step_ms, num_msc=13)
  1236. features.update(self.process_matrix(msc, 'MSC'))
  1238. # Other spectral features...
  1239. # (Similar to existing code in extract_spectral_features method)
  1240. except Exception as e:
  1241. self.logger.error(f"Error processing spectral features for {identifier}: {e}")
  1243. elif feature_type == 'complexity':
  1244. # Extract complexity features from data
  1245. try:
  1246. # Higuchi Fractal Dimension
  1247. HFD = calculate_hfd_per_frame(data, fs, window_length_ms, window_step_ms, 10)
  1248. features.update(self.process_row(HFD, 'HFD'))
  1250. # Other complexity features...
  1251. # (Similar to existing code in extract_complexity_features method)
  1252. except Exception as e:
  1253. self.logger.error(f"Error processing complexity features for {identifier}: {e}")
  1255. elif feature_type == 'frequency':
  1256. # Extract frequency features from data
  1257. try:
  1258. # Fundamental Frequency (F0)
  1259. F0 = get_pitch(data, fs, window_length_ms, window_step_ms)
  1260. F0_valid = F0[~np.isnan(F0)]
  1261. features.update(self.process_row(F0_valid, 'F0'))
  1263. # Other frequency features...
  1264. # (Similar to existing code in extract_frequency_features method)
  1265. except Exception as e:
  1266. self.logger.error(f"Error processing frequency features for {identifier}: {e}")
  1268. elif feature_type == 'intensity':
  1269. # Extract intensity features from data
  1270. try:
  1271. # Root Mean Square amplitude
  1272. RMS = rms_amplitude(data, fs, window_length_ms, window_step_ms)
  1273. features.update(self.process_row(RMS, 'RMS'))
  1275. # Other intensity features...
  1276. # (Similar to existing code in extract_intensity_features method)
  1277. except Exception as e:
  1278. self.logger.error(f"Error processing intensity features for {identifier}: {e}")
  1280. elif feature_type == 'voice_quality':
  1281. # Extract voice quality features from data
  1282. try:
  1283. # Shimmer
  1284. SHIMMER = analyze_audio_shimmer(data, fs, window_length_ms, window_step_ms)
  1285. features.update(self.process_row(SHIMMER, 'SHIMMER'))
  1287. # Other voice quality features...
  1288. # (Similar to existing code in extract_voice_quality_features method)
  1289. except Exception as e:
  1290. self.logger.error(f"Error processing voice quality features for {identifier}: {e}")
  1292. return features
  1293. except Exception as e:
  1294. self.logger.error(f"Error processing {feature_type} features from audio data: {e}")
  1295. return {}
  1297. # Process each feature type
  1298. for feature_type in features_to_calculate:
  1299. if feature_type == 'all':
  1300. # Handle 'all' feature type
  1301. all_features = process_audio_with_feature('all')
  1302. result.update(all_features)
  1303. break
  1304. else:
  1305. # Handle specific feature type
  1306. features = process_audio_with_feature(feature_type)
  1307. result.update(features)
  1309. self.logger.info("Extracted %d features total for %s", len(result), identifier)
  1310. return result
  1312. def _organize_features_by_group(self, features: Dict[str, Any]) -> Dict[str, Dict[str, Any]]:
  1313. """
  1314. Organize features by their group
  1316. Parameters:
  1317. -----------
  1318. features : Dict[str, Any]
  1319. Dictionary of features
  1321. Returns:
  1322. --------
  1323. Dict[str, Dict[str, Any]]
  1324. Dictionary of features organized by group
  1325. """
  1326. if not features:
  1327. return {}
  1329. # Group features by their category
  1330. feature_groups = self._group_features(list(features.keys()))
  1332. # Create output dictionary organized by groups
  1333. grouped_features = {}
  1335. for group_name, feature_names in feature_groups.items():
  1336. group_dict = {}
  1337. for name in feature_names:
  1338. if name in features:
  1339. group_dict[name] = features[name]
  1341. if group_dict: # Only include groups with features
  1342. grouped_features[group_name] = group_dict
  1344. # Check for any ungrouped features and add them to 'Other'
  1345. all_grouped_names = [name for group in feature_groups.values() for name in group]
  1346. ungrouped_features = {name: value for name, value in features.items()
  1347. if name not in all_grouped_names}
  1349. if ungrouped_features:
  1350. if 'Other' not in grouped_features:
  1351. grouped_features['Other'] = {}
  1352. grouped_features['Other'].update(ungrouped_features)
  1354. return grouped_features
  1356. def _flatten_grouped_features(self, grouped_features: Dict[str, Dict[str, Any]]) -> Dict[str, Any]:
  1357. """
  1358. Flatten a dictionary of grouped features into a single-level dictionary
  1360. Parameters:
  1361. -----------
  1362. grouped_features : Dict[str, Dict[str, Any]]
  1363. Dictionary of features organized by group
  1365. Returns:
  1366. --------
  1367. Dict[str, Any]
  1368. Flattened dictionary of features
  1369. """
  1370. flattened = {}
  1371. for group_name, group_dict in grouped_features.items():
  1372. for feature_name, feature_value in group_dict.items():
  1373. flattened[feature_name] = feature_value
  1374. return flattened
  1376. def _save_features_to_csv(self, features: Dict[str, Any], csv_path: str) -> None:
  1377. """
  1378. Save extracted features to a CSV file
  1380. Parameters:
  1381. -----------
  1382. features : Dict[str, Any]
  1383. Dictionary of extracted features
  1384. csv_path : str
  1385. Path to save the CSV file
  1386. """
  1387. try:
  1388. with open(csv_path, 'w', newline='') as csvfile:
  1389. writer = csv.writer(csvfile)
  1390. writer.writerow(['Feature', 'Value'])
  1392. for feature_name, feature_value in features.items():
  1393. # Handle different types of feature values
  1394. if isinstance(feature_value, (int, float)) or feature_value is None:
  1395. writer.writerow([feature_name, feature_value])
  1396. elif isinstance(feature_value, (list, np.ndarray)):
  1397. # For arrays, convert to string representation
  1398. if isinstance(feature_value, np.ndarray):
  1399. # Convert to Python list for better CSV compatibility
  1400. feature_list = feature_value.tolist()
  1401. else:
  1402. feature_list = feature_value
  1404. # Limit array size in CSV to prevent excessive file sizes
  1405. if len(feature_list) > 100:
  1406. self.logger.warning(f"Feature {feature_name} has {len(feature_list)} elements, truncating to 100 for CSV output")
  1407. feature_list = feature_list[:100]
  1409. # Join array elements with commas for CSV
  1410. writer.writerow([feature_name, str(feature_list)])
  1411. else:
  1412. # For other types, use string representation
  1413. writer.writerow([feature_name, str(feature_value)])
  1415. self.logger.info(f"Saved features to {csv_path}")
  1416. except Exception as e:
  1417. self.logger.error(f"Error saving features to CSV {csv_path}: {e}")
  1419. def _group_features(self, feature_names: List[str]) -> Dict[str, List[str]]:
  1420. """
  1421. Group features by their category based on name patterns
  1423. Parameters:
  1424. -----------
  1425. feature_names : List[str]
  1426. List of feature names to group
  1428. Returns:
  1429. --------
  1430. Dict[str, List[str]]
  1431. Dictionary mapping category names to lists of feature names
  1432. """
  1433. groups = {}
  1435. # Define common feature prefixes and their corresponding groups
  1436. # More specific prefixes first
  1437. prefix_groups = {
  1438. 'LOG_MEL_SPECTROGRAM': 'Spectral/Mel',
  1439. 'MFCC': 'Spectral/Cepstral',
  1440. 'LPCC': 'Spectral/Cepstral',
  1441. 'LPC': 'Spectral/LPC',
  1442. 'LSP': 'Spectral/LSP',
  1443. 'PLP': 'Spectral/PLP',
  1444. 'F0': 'Pitch',
  1445. 'F1': 'Formants',
  1446. 'F2': 'Formants',
  1447. 'F3': 'Formants',
  1448. 'Jitter': 'Voice Quality/Perturbation',
  1449. 'Shimmer': 'Voice Quality/Perturbation',
  1450. 'APQ': 'Voice Quality/Perturbation',
  1451. 'HNR': 'Voice Quality/Harmonicity',
  1452. 'NHR': 'Voice Quality/Harmonicity',
  1453. 'HARMONICITY': 'Voice Quality/Harmonicity',
  1454. 'CPP': 'Voice Quality/CPP',
  1455. 'HAMMARBERG_INDEX': 'Voice Quality/Spectral Tilt',
  1456. 'ALPHA_RATIO': 'Voice Quality/Spectral Tilt',
  1457. 'MSC': 'Spectral/Modulation',
  1458. 'CENTRIODS': 'Spectral/Shape',
  1459. 'LTAS': 'Spectral/Shape',
  1460. 'ENVELOPE': 'Spectral/Shape',
  1461. 'RMS': 'Intensity/Amplitude',
  1462. 'PEAK': 'Intensity/Amplitude',
  1463. 'Amplitude_Range': 'Intensity/Amplitude',
  1464. 'SPL': 'Intensity/SPL',
  1465. 'STE': 'Intensity/Energy',
  1466. 'INTENSITY': 'Intensity/Energy',
  1467. 'HFD': 'Complexity',
  1468. 'FREQ_ENTROPY': 'Complexity',
  1469. 'AMP_ENTROPY': 'Complexity',
  1470. 'ZCR': 'Complexity',
  1471. 'voiceProb': 'Speech Activity',
  1472. 'relative_sentence_duration': 'Speech Fluency/Timing',
  1473. 'regularity': 'Speech Fluency/Regularity',
  1474. 'PVI': 'Speech Fluency/Regularity'
  1475. # Add more specific groups as needed
  1476. }
  1478. # Assign each feature to a group
  1479. for name in feature_names:
  1480. assigned = False
  1481. # Check specific prefixes first
  1482. for prefix, group in prefix_groups.items():
  1483. # Match start or common statistical patterns
  1484. if name.startswith(prefix) or \
  1485. f'_{prefix}_sma' in name or \
  1486. f'_{prefix}_de' in name:
  1487. if group not in groups:
  1488. groups[group] = []
  1489. groups[group].append(name)
  1490. assigned = True
  1491. break
  1493. # Assign to broader category if no specific match
  1494. if not assigned:
  1495. broad_category = 'Other'
  1496. if 'spectral' in name.lower() or 'spec' in name.lower(): broad_category = 'Spectral/Other'
  1497. elif 'voice' in name.lower() or 'hnr' in name.lower() or 'jitter' in name.lower() or 'shimmer' in name.lower(): broad_category = 'Voice Quality/Other'
  1498. elif 'freq' in name.lower() or 'pitch' in name.lower() or 'formant' in name.lower(): broad_category = 'Frequency/Other'
  1499. elif 'intens' in name.lower() or 'loud' in name.lower() or 'amp' in name.lower() or 'spl' in name.lower() or 'peak' in name.lower() or 'rms' in name.lower(): broad_category = 'Intensity/Other'
  1500. elif 'complex' in name.lower() or 'entropy' in name.lower() or 'hfd' in name.lower(): broad_category = 'Complexity/Other'
  1501. elif 'fluency' in name.lower() or 'rhythm' in name.lower() or 'pause' in name.lower() or 'speech' in name.lower(): broad_category = 'Timing/Fluency/Other'
  1503. if broad_category not in groups:
  1504. groups[broad_category] = []
  1505. groups[broad_category].append(name)
  1507. return groups

AcousticFeatureExtractor.py at commit 0707335, no license · at the source

Overview

Authors: Maryam Zolnoori1,2,3,4,5, Elyas Esmaeili6, Mehdi Naserian6, Ali Zolnour6, Sina Rashidi6, Tahoura Morovati6, Hossein Azadmaleki6, Zhihong Zhang3, James M. Noble7, Margaret V. McDonald4
ORCID iDs: Maryam Zolnoori
  1. Columbia University Irving Medical Center,New York, NY 10027 USA
  2. School of Nursing, Columbia University,New York, NY 10027 USA
  3. Data Science Institute, Columbia University,New York, NY 10027 USA
  4. Center for Home Care Policy & Research, VNS Health,New York, NY 10017 USA
  5. Columbia University Irving Medical Center,560 W 168 St, New York, NY 10032 USA
  6. Independent Researcher, New York, USA
  7. Department of Neurology, Taub Institute for Research on Alzheimer’s Disease and the Aging Brain, GH Sergievsky Center, Columbia University,New York, NY 10032 USA
Institutions: Columbia University Irving Medical Center (United States); Columbia University (United States); VNS Health (United States)
Journal: Health information science and systems, volume 14, issue 1, article 77
Dates: received 8 March 2025; accepted 17 June 2026; published online 24 July 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1007/s13755-026-00468-5 · PMID 42502354 · PMCID PMC13400534 · OpenAlex W7170325869
Open access: hybrid, a free copy (OpenAlex)
Status: code verified
Methods: Statistics, Smoothing, state filtering, decompositions, Machine learning, Preprocessing, Spectral & time-frequency, Connectivity, Complexity
Topic: Dementia and Cognitive Impairment Research (Psychiatry and Mental health, Medicine), according to OpenAlex
Funding: National Institute on Aging (K99AG076808)
Citations: not cited yet (Europe PMC); 145 references in the paper

Abstract

Background: Early detection of cognitive impairment remains a critical public health challenge. While biomarkers such as neuroimaging and cerebrospinal fluid analyses offer high sensitivity, their limited accessibility hampers widespread screening, especially in underserved settings. Speech-based markers have emerged as promising, noninvasive indicators of cognitive decline.

Objective: To develop and validate SpeechDETECT, an end-to-end speech-processing pipeline that captures fine-grained acoustic and temporal markers of cognitive impairment and provides interpretable outputs suitable for large-scale screening.

Methods: SpeechDETECT comprises six modules: (1) noise reduction / amplitude normalization; (2) an eight-domain voice-analysis framework (e.g., frequency parameters, speech fluency); (3) 50 ms segment-level feature extraction; (4) feature visualization; (5) dimensionality reduction / selection (Joint Mutual Information Maximization, LassoNet, PCA); and (6) classifier training with SHapley Additive exPlanations (SHAP). Performance was benchmarked against six acoustic toolkits (e.g., GeMAPS) on two English datasets: the DementiaBank Pitt corpus (train = 166, test = 71) with single cookie-theft picture description task and NIA PREPARE Phase 2 corpus (train = 1 064, test = 267) with multiple speech tasks.

Results: A Multi-Layer Perceptron trained on PCA-derived SpeechDETECT features achieved an F1-score = 0.81% and AUC-ROC = 0.80 on the Pitt test set, outperforming the best competing toolkit (AUC = 0.76). On the PREPARE test set—comprising ≤ 30 s recordings from four speech tasks—the same model attained F1 ≈ 0.67% and AUC-ROC = 0.70, demonstrating good generalizability. Cumulative-gains analysis showed that screening the top 40% of ranked participants captured ~ 70% of cognitively-impaired (CI) cases in Pitt and ~ 63% in PREPARE. SHAP revealed speech-fluency metrics (hesitation rate, pause ratio) and high-frequency formant dynamics as the most discriminative features.

Conclusion: SpeechDETECT delivers accurate (AUC up to 0.80) and interpretable detection of early cognitive impairment across both structured and multi-task speech settings. Its fully automated, domain-informed approach enables scalable, speech-based screening and provides a foundation for multimodal systems that combine acoustic markers with clinical or biomarker data to further improve diagnostic precision. The SpeechDETECT toolkit is openly available on GitHub at https://github.com/SpeechCARE/SpeechDETECT-Toolkit for researchers and clinicians. A demo tutorial video showing pipeline usage is available at https://github.com/SpeechCARE/SpeechDETECT-Toolkit/blob/main/SpeechDETECT.mp4.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 14 matches between paragraphs and lines of code.

SpeechCARE/SpeechDETECT-Toolkit

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: 0707335a7e153736c9c06af7b1a1a8b0779ef6d1, 2 August 2026
Languages: Python (13)
Size: 18 files, 13 scripts
Software Heritage: not archived
Found in: “Code availability”
Holds: README, environment (requirements.txt, setup.py)
Not found: license file, CITATION.cff, tests, continuous integration, documentation
Tools: NumPy (9 files), SciPy (7 files), Matplotlib (1 file), PyTorch (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
14 files
At the source: github.com/SpeechCARE/SpeechDETECT-Toolkit

Code availability

The codes used for data analysis is publicly available here: https://github.com/SpeechCARE/SpeechDETECT-Toolkit.

Reproduced under the paper's license (CC BY), from the paper cited above.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 13 scripts, each with its path and the digest of its content;
  • 14 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability

The dataset used for this study is from DementiaBank dataset, the largest publicly available benchmark with audio recordings for the picture-description task.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 10 authors, 1 funder, 93 references.

Cite

This paper

Zolnoori, M., Esmaeili, E., Naserian, M., Zolnour, A., Rashidi, S., Morovati, T., Azadmaleki, H., Zhang, Z., Noble, J. M., & McDonald, M. V. (2026). SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes. Health information science and systems, 14(1), 77. https://doi.org/10.1007/s13755-026-00468-5

BibTeX

@article{zolnoori2026speechdetect,
author = {Zolnoori, Maryam and Esmaeili, Elyas and Naserian, Mehdi and Zolnour, Ali and Rashidi, Sina and Morovati, Tahoura and Azadmaleki, Hossein and Zhang, Zhihong and Noble, James M. and McDonald, Margaret V.},
title = {{SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes}},
journal = {Health information science and systems},
year = {2026},
month = jul,
volume = {14},
number = {1},
pages = {77},
publisher = {Springer},
issn = {2047-2501},
doi = {10.1007/s13755-026-00468-5},
url = {https://doi.org/10.1007/s13755-026-00468-5},
pmid = {42502354},
pmcid = {PMC13400534}
}

RIS

TY - JOUR
AU - Zolnoori, Maryam
AU - Esmaeili, Elyas
AU - Naserian, Mehdi
AU - Zolnour, Ali
AU - Rashidi, Sina
AU - Morovati, Tahoura
AU - Azadmaleki, Hossein
AU - Zhang, Zhihong
AU - Noble, James M.
AU - McDonald, Margaret V.
TI - SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes
T2 - Health information science and systems
J2 - Health Inf Sci Syst
PY - 2026
DA - 2026/07/24
VL - 14
IS - 1
SP - 77
SN - 2047-2501
PB - Springer
DO - 10.1007/s13755-026-00468-5
UR - https://doi.org/10.1007/s13755-026-00468-5
LA - en
ER -

CSL-JSON

{
"id": "10.1007/s13755-026-00468-5",
"type": "article-journal",
"title": "SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes",
"container-title": "Health information science and systems",
"author": [
{
"family": "Zolnoori",
"given": "Maryam"
},
{
"family": "Esmaeili",
"given": "Elyas"
},
{
"family": "Naserian",
"given": "Mehdi"
},
{
"family": "Zolnour",
"given": "Ali"
},
{
"family": "Rashidi",
"given": "Sina"
},
{
"family": "Morovati",
"given": "Tahoura"
},
{
"family": "Azadmaleki",
"given": "Hossein"
},
{
"family": "Zhang",
"given": "Zhihong"
},
{
"family": "Noble",
"given": "James M."
},
{
"family": "McDonald",
"given": "Margaret V."
}
],
"container-title-short": "Health Inf Sci Syst",
"volume": "14",
"issue": "1",
"page": "77",
"DOI": "10.1007/s13755-026-00468-5",
"PMID": "42502354",
"PMCID": "PMC13400534",
"ISSN": "2047-2501",
"publisher": "Springer",
"URL": "https://doi.org/10.1007/s13755-026-00468-5",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
24
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1002/alz.71365 [code]
Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures.
Journal: Alzheimer's & dementia : the journal of the Alzheimer's Association
In common: SciPy, Matplotlib, NumPy, 2 references
[2] doi:10.1038/s41514-026-00390-w [code]
Mild cognitive impairment cases affect the predictive power of Alzheimer's disease diagnostic models using routine clinical variables.
Journal: npj aging
In common: SciPy, Matplotlib, NumPy, 2 references
[3] doi:10.1002/advs.77003 [code]
SemanticST: A Scalable Multi-Contextual Graph Learning Framework for Uncovering Spatial Niches and Robust Multi-Sample Integration in Spatial Transcriptomics.
Journal: Advanced science (Weinheim, Baden-Wurttemberg, Germany)
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[4] doi:10.3390/s26103065 [code]
Subject-Wise Depression Screening from Eight-Channel Resting-State EEG Using Asymmetry-Aware Spectral Features and Connectivity Ablation.
Journal: Sensors (Basel, Switzerland)
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[5] doi:10.1371/journal.pgen.1012242 [code]
Wiz regulates clustered protocadherin genes by restricting CTCF/cohesin loop extrusion in a genomic-distance biased manner.
Journal: PLoS genetics
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[6] doi:10.1371/journal.pone.0351872 [code]
Decoding visual object recognition from EEG signals.
Journal: PloS one
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[7] doi:10.1038/s41398-026-04101-7 [code]
Using deep learning to identify brain networks mediating cognitive and motor impairments in alcohol use disorder.
Journal: Translational psychiatry
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[8] doi:10.1002/mrm.70431 [code]
SelExNet: A Self-Supervised Physics-Informed Framework for Multi-Channel Joint RF and Gradient Waveform Optimization in 2D Spatially Selective Excitation.
Journal: Magnetic resonance in medicine
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[9] doi:10.3389/fsysb.2026.1873899 [code]
A systems microbiology framework for reproducible multi-dataset omics integration with application to long COVID.
Journal: Frontiers in systems biology
In common: PyTorch, SciPy, Matplotlib, 1 other tool, 1 reference
[10] doi:10.1093/nar/gkag621 [code]
Optimal gene panel selection for targeted spatial transcriptomics experiments.
Journal: Nucleic acids research
In common: PyTorch, Matplotlib, NumPy, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.