Deep Learning-Based Temporal Gait Analysis Using a Smartphone IMU in Older Adults with and Without Non-Specific Low Back Pain.

Code ↔ Paper

4 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 4 matches

[1] § 2. Materials and Methods › 2.4. Model Training and Evaluation ↔ notebooks/03_RNN_eval.ipynb, lines 760–817 · score 0.57 · IC FO macro, training strategy, weighted, architecture, expansion, fold
[2] § 2. Materials and Methods › 2.6. Statistical Analysis ↔ notebooks/03_RNN_eval.ipynb, lines 1314–1393 · score 0.55 · square error, gait event detection, absolute error, root
[3] § 2. Materials and Methods › 2.4. Model Training and Evaluation ↔ notebooks/03_RNN-Copy1.ipynb, lines 117–242 · score 0.51 · PyTorch, GPU, epochs, log, loss, batch
[4] § 2. Materials and Methods › 2.4. Model Training and Evaluation ↔ notebooks/03_RNN-Copy2.ipynb, lines 120–245 · score 0.51 · PyTorch, GPU, epochs, log, loss, batch

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The paper is loaded by your browser, with JavaScript: read it at doi.org or on Europe PMC.

The authors' code

Jupyter notebook · 1,600 lines · 51 KB · other · 2 matches

# %%
import os
from pathlib import Path
%matplotlib inline
%load_ext autoreload
%autoreload 2
import seaborn as sns
import pickle
import time
# %%
from datetime import datetime
from glob import glob
import lightning as pl
import numpy as np
import pandas as pd
import wandb
from gait_ml.data.datamodule import GaitDataModule
from gait_ml.model.litmodel import LitSeq2Seq
from lightning.pytorch.callbacks import LearningRateMonitor, ModelCheckpoint
from lightning.pytorch.loggers import WandbLogger
from sklearn.model_selection import StratifiedKFold, train_test_split
from hydra import initialize, compose
import hydra
from omegaconf import DictConfig
import torch
from gait_ml import evaluate
from torchmetrics.classification import (
MulticlassPrecision,
MulticlassRecall,
BinaryPrecision,
BinaryRecall,
Accuracy,
ConfusionMatrix,
MulticlassConfusionMatrix,
MulticlassF1Score,
F1Score,
)
import matplotlib.pyplot as plt
def calculate_gait_mae(
ground_truth: np.ndarray, prediction: np.ndarray, tolerance: int = 20
):
"""
Matches gait events and computes overall and per-class MAE in one pass.
Args:
ground_truth: Array of ground truth labels.
prediction: Array of predicted labels.
tolerance: Max distance to consider a match.
Returns:
A tuple containing:
- overall_mae (float): The MAE across all matched events.
- per_class_mae (dict): A dictionary mapping class labels to their MAE.
"""
gt_indices = np.where(ground_truth > 0)[0]
available_preds = list(np.where(prediction > 0)[0])
# Store tuples of (label, absolute_error) for each successful match
errors_by_class = []
# Greedily match each ground truth event to the closest prediction
for gt_idx in gt_indices:
gt_label = ground_truth[gt_idx]
best_dist = float("inf")
best_match_idx = -1
# Find the closest available prediction of the same class within tolerance
for pred_idx in available_preds:
if prediction[pred_idx] == gt_label:
dist = abs(gt_idx - pred_idx)
if dist <= tolerance and dist < best_dist:
best_dist = dist
best_match_idx = pred_idx
# If a match is found, record its error and remove it from the pool
if best_match_idx != -1:
error = abs(gt_idx - best_match_idx)
errors_by_class.append((gt_label, error))
available_preds.remove(best_match_idx)
if not errors_by_class:
return np.nan, {}
# --- Calculate Final Metrics ---
# Overall MAE is the mean of all collected errors
all_errors = [err for lbl, err in errors_by_class]
overall_mae = np.mean(all_errors)
# Per-class MAE is calculated by grouping errors by label
unique_labels = sorted(np.unique([lbl for lbl, err in errors_by_class]))
per_class_mae = {
int(label): np.mean([err for lbl, err in errors_by_class if lbl == label])
for label in unique_labels
}
return overall_mae, per_class_mae
# @hydra.main(config_path="../configs", config_name="train_config", version_base="1.3")
def eval(data_set: str, model_fpath: str, fold: int, return_preds_targets=None):
with initialize(config_path="../configs", job_name="train", version_base="1.3"):
config = compose(config_name="train_config")
pl.seed_everything(config.general.random_state)
all_files = glob(config.general.data_path, recursive=True)
all_files = np.sort(all_files)
print(f"Processing: {len(all_files)} samples")
ids = [int(i.split("/")[-1].split("_")[0]) for i in all_files]
group_df = pd.read_csv(config.general.group_file, index_col="ID")
group_df.columns = ["group"]
group_df = group_df[group_df.group.notna()]
group_df.replace("h", 0, inplace=True)
group_df.replace("p", 1, inplace=True)
grouping = group_df.loc[ids].group.values
skf = StratifiedKFold(
n_splits=config.general.n_splits,
shuffle=True,
random_state=config.general.random_state,
)
# Outer loop for K-Fold cross-validation
# This loop creates the primary TEST set for each fold.
for cur_fold, (train_val_index, test_index) in enumerate(
skf.split(np.arange(len(ids)).reshape(-1, 1), grouping)
):
if cur_fold != fold:
continue
print("cur_fold", cur_fold)
print("input fold", fold)
print(
f"=============== FOLD {cur_fold + 1}/{config.general.n_splits} ================"
)
print("current_test set:", test_index)
# Split data into a temporary training+validation set and the final test set
X_train_val, X_test = (
np.arange(len(ids))[train_val_index],
np.arange(len(ids))[test_index],
)
print(f"X_train_val:", X_train_val)
print(f"train_val_index:", train_val_index)
print(f"X_test:", X_test)
print(f"test_index:", test_index)
y_train_val, y_test = grouping[train_val_index], grouping[test_index]
X_train, X_val, y_train, y_val = train_test_split(
X_train_val,
y_train_val,
test_size=0.25,
stratify=y_train_val,
random_state=1,
)
model = LitSeq2Seq(
input_dim=config.model.input_dim,
output_dim=config.model.output_dim,
hidden_dim=config.model.hidden_dim,
num_layers=config.model.num_layers,
dropout_prob=config.model.dropout_prob,
learning_rate=config.model.learning_rate,
teacher_forcing_ratio=config.model.teacher_forcing_ratio,
)
current_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
run_name = f"GRU-expandlabel{config.data.expand_labels}_{current_time}"
checkpoint_callback = ModelCheckpoint(
monitor=config.training.monitor_metric,
mode=config.training.monitor_mode,
save_top_k=config.training.save_top_k,
dirpath=f"{config.general.project_name}/{run_name}/checkpoints/",
filename="model-{epoch:02d}-{val_f1score:.2f}",
)
checkpoint_callback.best_model_path = model_fpath
print(f"Best model: {checkpoint_callback.best_model_path}")
device_ = (
torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
)
best_model = LitSeq2Seq.load_from_checkpoint(
checkpoint_callback.best_model_path
).to(device_)
testset_f1_scores = []
testset_pc_f1_scores = []
testset_pc_precision = []
testset_pc_recall = []
testset_mae = []
testset_pc_mae = []
testset_cm = []
all_preds = []
all_targets = []
if data_set == "test":
cur_set_to_test = X_test
else:
cur_set_to_test = X_val
print("X_test:", X_test)
print("X_val:", X_val)
group_assigment = []
counter = 0
for curr_test_idx in cur_set_to_test.tolist():
print(
f" =============== Currently Evaluating curr_test_idx: {curr_test_idx} =============== "
)
if counter == 0:
test_datamodule = GaitDataModule(
all_files,
batch_size=1024,
window_size=config.data.window_size,
step_size=config.data.window_size,
train_idx=X_train.squeeze(),
val_idx=X_val.squeeze()[
:1
], # We load validation/test set via the test_idx here
test_idx=[curr_test_idx],
expand_labels=0,
acc_sheet_name="Linear Accelerometer",
num_workers=16,
zscale=True,
)
test_datamodule.setup(data_set)
trainset_stats = test_datamodule.train_dataset.zscale_stats
counter += 1
test_datamodule = GaitDataModule(
all_files,
batch_size=1024,
window_size=config.data.window_size,
step_size=config.data.window_size,
train_idx=X_train.squeeze(),
val_idx=X_val.squeeze()[
:1
], # We load validation/test set via the test_idx here
test_idx=[curr_test_idx],
expand_labels=0,
acc_sheet_name="Linear Accelerometer",
num_workers=16,
zscale=True,
zscale_stats=trainset_stats,
)
test_datamodule.setup(data_set)
test_dataloader = test_datamodule.test_dataloader()
# if data_set == "test":
# test_dataloader = test_datamodule.test_dataloader()
# elif data_set == "val":
# test_dataloader = test_datamodule.val_dataloader()
# else:
# raise ValueError(f"Not supported -> {data_set}")
best_model.eval()
# Benchmark single-window inference
sample_x, sample_y = next(iter(test_dataloader))
sample_x = sample_x[:1].to(device_) # shape: [1, 256, 6]
sample_y = sample_y[:1].to(device_)
# Warm-up
with torch.inference_mode():
for _ in range(20):
_ = best_model(
sample_x,
sample_y,
teacher_forcing_ratio=0.0,
)
if device_.type == "cuda":
torch.cuda.synchronize()
# Timed runs
n_runs = 100
inference_times = []
with torch.inference_mode():
for _ in range(n_runs):
if device_.type == "cuda":
torch.cuda.synchronize()
start = time.perf_counter()
_ = best_model(
sample_x,
sample_y,
teacher_forcing_ratio=0.0,
)
if device_.type == "cuda":
torch.cuda.synchronize()
inference_times.append(time.perf_counter() - start)
inference_times = np.array(inference_times) * 1000 # ms
print(f"Mean inference time: {inference_times.mean():.3f} ms/window")
print(f"Median inference time: {np.median(inference_times):.3f} ms/window")
print(f"SD inference time: {inference_times.std():.3f} ms")
with torch.no_grad():
pred = []
target = []
for sample_input, sample_target in test_dataloader:
sample_input = sample_input.to(device_)
sample_target = sample_target.to(device_)
print(sample_input.shape)
predicted_output = best_model(
sample_input, sample_target, teacher_forcing_ratio=0.0
)
pred.append(predicted_output)
target.append(sample_target)
cur_pred = torch.concat(pred).reshape(-1, 3)
cur_pred = torch.nn.functional.softmax(cur_pred, dim=-1).argmax(1)
cur_target = torch.concat(target).reshape(-1)
reshaped_input = (
sample_input.reshape(-1, sample_input.shape[-1]).cpu().numpy()
)
print("Pred", cur_pred.shape, "Target", cur_target.shape)
# cm = MulticlassConfusionMatrix(num_classes=3, normalize="true").to(device_)
# cm.update(cur_pred, cur_target)
# fig_, ax_ = cm.plot()
# plt.show()
ALIGN_TOLERANCE = 3
merged_preds = evaluate.merge_clustered_events(cur_pred.cpu().numpy())
merged_targets = evaluate.merge_clustered_events(
cur_target.cpu().numpy()
)
aligned_preds = evaluate.align_events(
merged_targets, merged_preds, ALIGN_TOLERANCE
)
cm = MulticlassConfusionMatrix(num_classes=3, normalize="true")
cm.update(torch.tensor(aligned_preds), torch.tensor(merged_targets))
fig_, ax_ = cm.plot()
plt.show()
unnormalized_cm = MulticlassConfusionMatrix(
num_classes=3, normalize=None
)
unnormalized_cm.update(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
ucm_tensor = unnormalized_cm.compute()
testset_cm.append(ucm_tensor)
print(ucm_tensor)
num_classes = 3
f1_macro = MulticlassF1Score(num_classes=num_classes, average="macro")
f1_macro_score = f1_macro(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
print(f"Macro F1-Score: {f1_macro_score.item():.4f} ✨")
pc_f1score = MulticlassF1Score(num_classes=num_classes, average="none")
perclass_f1scores = pc_f1score(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
testset_pc_f1_scores.append(perclass_f1scores)
print(f"perclass_f1scores:", perclass_f1scores)
pc_precision = MulticlassPrecision(
num_classes=num_classes, average="none"
)
perclass_precision = pc_precision(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
testset_pc_precision.append(perclass_precision)
print(f"perclass_precision:", perclass_precision)
pc_recall = MulticlassRecall(num_classes=num_classes, average="none")
perclass_recall = pc_recall(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
testset_pc_recall.append(perclass_recall)
print(f"perclass_recall:", perclass_recall)
f1_macro_score = f1_macro(
torch.tensor(aligned_preds), torch.tensor(merged_targets)
)
overall, per_class = calculate_gait_mae(merged_targets, merged_preds)
overall = overall.item() if not np.isnan(overall) else np.nan
print(f"MAE overall: {overall:.4f} ✨")
print(f"MAE per_class: {per_class} ✨")
testset_f1_scores.append(f1_macro_score)
testset_mae.append(overall)
testset_pc_mae.append(per_class)
group_assigment.append(grouping[curr_test_idx])
if return_preds_targets:
all_preds.append(aligned_preds)
all_targets.append(merged_targets)
# break
# break
return {
"testset_f1_scores": torch.tensor(testset_f1_scores).numpy(),
"testset_pc_f1_scores": torch.stack(testset_pc_f1_scores).numpy(),
"testset_pc_precision": torch.stack(testset_pc_precision).numpy(),
"testset_pc_recall": torch.stack(testset_pc_recall).numpy(),
"testset_mae": np.stack(testset_mae),
"testset_pc_mae": testset_pc_mae,
"testset_cm": torch.stack(testset_cm).numpy(),
"group": np.stack(group_assigment),
"all_preds": all_preds,
"all_targets": all_targets,
}
# %% [markdown]
# # 1. Evaluate Validation set
# %%
# Evaluate all best model on the validation set
num_folds = 5
# exp_labels = [2, 1, 4, 8]
exp_labels = [2]
data_set = "val" # using train here because we are loading all dataset anyway and using training set stats to normalize data
for i in range(num_folds):
for j in exp_labels:
print(i, j)
# curr_model_path = np.sort(glob(f"/home/geromevivar/projects/gait_ml/backpain/Fold{i+1}*expandlabel{j}*/*/*"))[-1]
# curr_model_path = np.sort(glob(f"/home/qivy00li/projects/gait_ml/backpain/RerunExp-Fold{i+1}*expandlabel{j}*/*/*"))[-1]
# curr_model_path = np.sort(glob(f"/home/qivy00li/projects/gait_ml/backpain/ZscaledRerunExp4-Fold{i+1}*expandlabel{j}*/*/*"))[-1]
##### MDPI
# LSTM MDPI
# curr_model_path = np.sort(glob(f"/home/qivy00li/projects/gait_ml/backpain_mdpi_runs/lstm/ZscaledRerunExp4-Fold{i+1}*expandlabel{j}*/*/*"))[-1]
# GRU/LSTM weighted loss
# curr_model_path = np.sort(glob(f"/home/qivy00li/projects/gait_ml/backpain_mdpi_runs/gru_lossweighted/ZscaledRerunExp4-Fold{i+1}*expandlabel{j}*/*/*"))[-1]
# Rerun for MDPI
curr_model_path = np.sort(
glob(
f"/home/qivy00li/projects/gait_ml/backpain_mdpi_runs/gru/ZscaledRerunExp4-Fold{i + 1}*expandlabel{j}*/*/*"
)
)[-1]
print(os.path.exists(curr_model_path), curr_model_path)
save_name = (
f"ZscaledRerunExp4-Fold{i + 1}_explabel{j}_{Path(curr_model_path).stem}.npz"
)
eval_output_dir = os.path.join(
"./mdpi_evals/", Path(curr_model_path).parents[1].name
)
save_name = os.path.join(eval_output_dir, save_name)
if not os.path.exists(eval_output_dir):
os.makedirs(eval_output_dir, exist_ok=True)
# if data_set == "test" or data_set == "val":
# save_name = f"{data_set}set_{save_name}"
if not os.path.exists(save_name):
print(f"=== Running model: {save_name} ===")
testset_results = eval(data_set=data_set, model_fpath=curr_model_path, fold=i)
# np.savez(save_name, **testset_results)
# %% [markdown]
# # 2. Plot
# %% [markdown]
# ### 2.1a - MDPI new plots
# %%
from pathlib import Path
import re
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy.stats import t
# ============================================================
# SETTINGS
# ============================================================
ROOT = Path("./mdpi_evals")
# 1 = ±10 ms
# 2 = ±20 ms
# 4 = ±40 ms
# 8 = ±80 ms
SELECTED_EXPANSION = 2
METRIC_KEY = "testset_f1_scores"
# ============================================================
# DEFINE THE SIX EXPERIMENT GROUPS
# Each pattern should find one folder per fold.
# ============================================================
experiments = [
{
"architecture": "GRU",
"strategy": "Point labels",
"pattern": "ZscaledRerunExp4-Fold*-GRU-expandlabel0_*",
"weighted": False,
},
{
"architecture": "GRU",
"strategy": "Weighted CE",
"pattern": "ZscaledRerunExp4-Fold*-gru-expandlabel0_*lossweighted=True",
"weighted": True,
},
{
"architecture": "GRU",
"strategy": "Label expansion",
"pattern": (f"ZscaledRerunExp4-Fold*-GRU-expandlabel{SELECTED_EXPANSION}_*"),
"weighted": False,
},
{
"architecture": "LSTM",
"strategy": "Point labels",
"pattern": "ZscaledRerunExp4-Fold*-lstm-expandlabel0_*",
"weighted": False,
},
{
"architecture": "LSTM",
"strategy": "Weighted CE",
"pattern": "ZscaledRerunExp4-Fold*-lstm-expandlabel0_*lossweighted=True",
"weighted": True,
},
{
"architecture": "LSTM",
"strategy": "Label expansion",
"pattern": (f"ZscaledRerunExp4-Fold*-lstm-expandlabel{SELECTED_EXPANSION}_*"),
"weighted": False,
},
]
# ============================================================
# LOAD ONE EVENT MACRO-F1 VALUE FROM EACH NPZ FILE
# ============================================================
def load_event_macro_f1(npz_path):
"""
Return one combined IC–FO F1 score for one fold.
Current implementation assumes METRIC_KEY contains only
the IC and FO F1 scores, or values whose overall mean is
the intended event macro-F1.
"""
with np.load(npz_path, allow_pickle=True) as results:
if METRIC_KEY not in results.files:
raise KeyError(
f"{METRIC_KEY!r} not found in {npz_path}.\n"
f"Available keys: {results.files}"
)
scores = np.asarray(
results[METRIC_KEY],
dtype=float,
)
# Use this when the array contains only IC and FO scores.
event_macro_f1 = float(np.nanmean(scores))
# If the array instead contains [non-event, IC, FO],
# replace the line above with:
#
# event_macro_f1 = float(np.nanmean(scores[..., 1:3]))
return event_macro_f1
# ============================================================
# FIND FOLD FOLDERS AND BUILD DATAFRAME
# ============================================================
rows = []
for experiment in experiments:
candidate_folders = sorted(ROOT.glob(experiment["pattern"]))
if (experiment["architecture"] == "LSTM") and (
experiment["strategy"] == "Point labels"
):
candidate_folders = [
i for i in candidate_folders if "lossweighted=True" not in i.name
]
# print(len(candidate_folders), experiment["pattern"])
# Point-label and weighted-CE folders share the same base
# pattern, so distinguish them using lossweighted=True.
selected_folders = []
for folder in candidate_folders:
folder_is_weighted = "lossweighted=true" in folder.name.lower()
if folder_is_weighted == experiment["weighted"]:
selected_folders.append(folder)
print(
f"{experiment['architecture']:4s} | "
f"{experiment['strategy']:16s} | "
f"{len(selected_folders)} folders"
)
for folder in selected_folders:
fold_match = re.search(
r"Fold(\d+)",
folder.name,
flags=re.IGNORECASE,
)
if fold_match is None:
print(f"Could not identify fold: {folder}")
continue
fold = int(fold_match.group(1))
npz_files = sorted(folder.glob("*.npz"))
if len(npz_files) == 0:
print(f"No NPZ file found in: {folder}")
continue
if len(npz_files) > 1:
print(f"Multiple NPZ files found in {folder}. Using: {npz_files[-1].name}")
npz_path = npz_files[-1]
macro_f1 = load_event_macro_f1(npz_path)
rows.append(
{
"Architecture": experiment["architecture"],
"Strategy": experiment["strategy"],
"Fold": fold,
"Macro-F1": macro_f1,
"File": str(npz_path),
}
)
df = pd.DataFrame(rows)
if df.empty:
raise RuntimeError("No results were loaded. Check ROOT and the folder patterns.")
df = df.sort_values(["Architecture", "Strategy", "Fold"]).reset_index(drop=True)
print("\nLoaded results:")
print(
df[
[
"Architecture",
"Strategy",
"Fold",
"Macro-F1",
]
].to_string(index=False)
)
# ============================================================
# CHECK THAT EACH CONDITION HAS FIVE FOLDS
# ============================================================
fold_counts = (
df.groupby(["Architecture", "Strategy"])["Fold"].nunique().rename("Number of folds")
)
print("\nFold counts:")
print(fold_counts)
missing_conditions = fold_counts[fold_counts != 5]
if not missing_conditions.empty:
raise RuntimeError(
"\nSome conditions do not contain exactly five folds:\n"
f"{missing_conditions}\n\n"
"Check the experiment folder patterns."
)
# ============================================================
# CALCULATE MEAN AND 95% CONFIDENCE INTERVAL
# ============================================================
summary = (
df.groupby(["Architecture", "Strategy"])["Macro-F1"]
.agg(["mean", "std", "count"])
.reset_index()
)
summary["standard_error"] = summary["std"] / np.sqrt(summary["count"])
summary["ci95"] = (
t.ppf(
0.975,
df=summary["count"] - 1,
)
* summary["standard_error"]
)
print("\nSummary:")
print(summary.to_string(index=False))
# ============================================================
# PLOT
# ============================================================
strategy_order = [
"Point labels",
"Weighted CE",
"Label expansion",
]
architecture_order = [
"GRU",
"LSTM",
]
x = np.arange(len(strategy_order))
architecture_offsets = {
"GRU": -0.07,
"LSTM": 0.07,
}
rng = np.random.default_rng(42)
fig, ax = plt.subplots(figsize=(9, 5.5))
for architecture in architecture_order:
architecture_summary = (
summary[summary["Architecture"] == architecture]
.set_index("Strategy")
.reindex(strategy_order)
)
x_architecture = x + architecture_offsets[architecture]
plot_result = ax.errorbar(
x_architecture,
architecture_summary["mean"],
yerr=architecture_summary["ci95"],
marker="o",
markersize=7,
linewidth=2,
capsize=5,
label=architecture,
)
line_color = plot_result.lines[0].get_color()
# Add individual fold values.
for position, strategy in enumerate(strategy_order):
fold_values = df[
(df["Architecture"] == architecture) & (df["Strategy"] == strategy)
]["Macro-F1"].to_numpy()
jitter = rng.normal(
loc=0,
scale=0.012,
size=len(fold_values),
)
ax.scatter(
np.full(
len(fold_values),
x_architecture[position],
)
+ jitter,
fold_values,
s=35,
alpha=0.65,
color=line_color,
zorder=3,
)
expansion_ms = SELECTED_EXPANSION * 10
ax.set_xticks(x)
ax.set_xticklabels(
[
"Point labels",
"Weighted CE",
f"Label expansion\n±{expansion_ms} ms",
]
)
ax.set_xlabel("Training strategy", fontsize=16, fontweight="bold")
ax.set_ylabel("IC–FO macro-F1", fontsize=16, fontweight="bold")
# ax.set_title(
# "Comparison of Architecture and Sparse-Label Strategy"
# )
ax.set_ylim(0, 1.02)
ax.legend(title="Architecture")
ax.grid(axis="y", alpha=0.25)
plt.xticks(fontsize=14)
plt.yticks(fontsize=14)
plt.tight_layout()
plt.savefig(
"architecture_strategy_comparison.png",
dpi=300,
bbox_inches="tight",
)
plt.show()
# %% [markdown]
# ## 2.1 Plot F1-score and MAE box plot validation set
# %%
# Evaluate all best model on the validation set
num_folds = 5
exp_labels = [0, 1, 2, 4, 8]
all_results = []
for i in range(num_folds):
for j in exp_labels:
# curr_model_path = np.sort(glob(f"/home/geromevivar/projects/gait_ml/backpain/Fold{i+1}*expandlabel{j}*/*/*"))[-1]
curr_model_path = np.sort(
glob(
f"/home/qivy00li/projects/gait_ml/backpain/RerunExp-Fold{i + 1}*expandlabel{j}*/*/*"
)
)[-1]
fname = f"valset_Fold{i + 1}_explabel{j}_{Path(curr_model_path).stem}.npz"
print(f" === Processing {fname} === ")
curr_results = np.load(fname, allow_pickle=True)
curr_results = {**curr_results}
curr_results["fold"] = i + 1
curr_results["explabel"] = j
all_results.append(curr_results)
all_results_df = pd.DataFrame(all_results)
print(all_results_df.head())
# Simplified Code for MEAN OF LIST LENGTHS
cur_feature_name = "testset_f1_scores"
cur_metric_name = "F1-Score"
f1_scores = all_results_df.groupby(["fold", "explabel"])[[cur_feature_name]].agg(
lambda s: s.apply(np.nanmean)
)
f1_scores = f1_scores.reset_index()
f1_scores.head()
# f1_scores.groupby(["explabel"]).mean()
# f1_scores.groupby(["explabel"]).std()
plot_df = pd.melt(f1_scores[f1_scores.explabel != 0], id_vars=["fold", "explabel"])
plot_df = plot_df.rename(columns={"explabel": "Expand", "value": cur_metric_name})
plt.figure(figsize=(10, 6))
g = sns.boxplot(data=plot_df, x="Expand", y=cur_metric_name)
# plt.title("Event Detection Performance on Validation Set (5-fold CV)")
g.set_xticklabels(
["\u00b1 10 [ms]", "\u00b1 20 [ms]", "\u00b1 40 [ms]", "\u00b1 80 [ms]"]
)
g.set_xlabel("Model's Label Expansion Setting")
g.set_title
# %%
# JNER version
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
# --- 1. JNER Style Setup ---
# Use standard sans-serif fonts (Arial/Helvetica)
plt.rcParams["font.family"] = "sans-serif"
plt.rcParams["font.sans-serif"] = ["Arial", "Helvetica", "DejaVu Sans"]
# --- [Your Data Loading Block Stays Here] ---
# (Assuming all_results_df is created as in your snippet)
# --- Data Preparation ---
cur_feature_name = "testset_f1_scores"
cur_metric_name = "F1-Score"
# Process data
f1_scores = all_results_df.groupby(["fold", "explabel"])[[cur_feature_name]].agg(
lambda s: s.apply(np.nanmean)
)
f1_scores = f1_scores.reset_index()
# Filter out 0ms if desired
plot_df = pd.melt(f1_scores[f1_scores.explabel != 0], id_vars=["fold", "explabel"])
plot_df = plot_df.rename(columns={"explabel": "Expand", "value": cur_metric_name})
# --- 2. Plotting for Publication ---
# Figure Size:
# Journals usually have columns ~3.5 inches wide.
# A width of 6-8 inches allows it to span two columns or be scaled down nicely.
plt.figure(figsize=(8, 5))
# Style: White background with grid is standard for scientific comparison
sns.set_style("whitegrid")
# Define explicit order to ensure labels match data
# (Assuming exp_labels 1, 2, 4, 8 correspond to 10, 20, 40, 80)
order_list = [1, 2, 4, 8]
g = sns.boxplot(
data=plot_df,
x="Expand",
y=cur_metric_name,
order=order_list, # Critical: Ensures X-axis is sorted correctly
width=0.5, # Thinner boxes look cleaner
linewidth=1.2, # Thicker lines for visibility in print
palette="Blues", # "Blues" is aesthetically pleasing and printer-safe
showfliers=False, # Optional: Hide outliers if they distract (check journal preference)
)
# --- 3. Formatting Axes ---
# Y-Axis: Use LaTeX rendering for F1-score if possible, or consistent text
# Note: Matplotlib can render simple math-like text without full LaTeX
plt.ylabel(r"$\mathbf{F_1}$-score", fontsize=16, fontweight="bold")
# X-Axis
plt.xlabel("Label Expansion Window", fontsize=16, fontweight="bold")
# Ticks: Use the ± symbol and standard units
# Mapping 1->10ms, 2->20ms, etc. based on your previous snippet
clean_labels = ["$\pm$10 ms", "$\pm$20 ms", "$\pm$40 ms", "$\pm$80 ms"]
g.set_xticklabels(clean_labels, fontsize=14)
plt.yticks(fontsize=14)
# Remove the top and right spines (cleaner look)
sns.despine()
# --- 4. Saving ---
# Remove title (It belongs in the LaTeX caption, not the image)
# plt.title("...")
plt.tight_layout()
# Save as PDF (Vector - Best) or PNG (Raster - High DPI)
plt.savefig("X_figures/f1_score_expansion_boxplot.pdf", bbox_inches="tight")
plt.savefig("X_figures/f1_score_expansion_boxplot.png", dpi=600, bbox_inches="tight")
plt.show()
# %%
# all_results_df[all_results_df.explabel==1].testset_f1_scores.apply(lambda x: x.mean()).mean()
# %%
import pandas as pd
import numpy as np
from scipy import stats
# --- Simplified Aggregation ---
# A single lambda function to calculate the 95% Confidence Interval bounds (CI)
# t.interval returns a tuple: (lower_bound, upper_bound)
def get_ci_bounds(series, confidence=0.95):
"""Calculates the 95% CI (lower, upper) for a Series."""
if series.empty:
return np.nan, np.nan
# stats.sem calculates the Standard Error of the Mean (sigma / sqrt(n))
sem = stats.sem(series, ddof=1)
# stats.t.interval computes the confidence interval
return stats.t.interval(confidence, len(series) - 1, loc=series.mean(), scale=sem)
# Assuming your DataFrame 'f1_scores' and column 'cur_feature_name' are defined.
decimals = 3
# 1. Aggregate the data (combining the CI calculation)
report_df = (
f1_scores.groupby("explabel")[cur_feature_name]
.agg(
mean_val="mean",
std_val="std",
ci_bounds=get_ci_bounds, # Uses the single function to get a tuple of bounds
count="size",
)
.reset_index()
)
# 2. Split the CI tuple into separate columns for easier formatting
report_df[["ci_lower", "ci_upper"]] = pd.DataFrame(
report_df["ci_bounds"].tolist(), index=report_df.index
)
report_df = report_df.drop(columns=["ci_bounds"])
# --- Simplified Formatting (Using f-strings and round) ---
# 3. Create the formatted columns using vectorization (not row-wise apply) where possible
# Note: Using .round() before f-string formatting ensures correct precision.
# Mean +/- Std Column
report_df["Mean_Std"] = (
report_df["mean_val"].round(decimals).astype(str)
+ " $\\pm$ "
+ report_df["std_val"].round(decimals).astype(str)
)
# Confidence Interval Column
report_df["95% CI"] = (
"["
+ report_df["ci_lower"].round(decimals).astype(str)
+ ", "
+ report_df["ci_upper"].round(decimals).astype(str)
+ "]"
)
# 4. Select and rename final columns for publication
publication_table = report_df[["explabel", "count", "Mean_Std", "95% CI"]]
publication_table.columns = [
"Experiment Label (Group)",
"N",
"Mean $\\pm$ Std. Dev.",
"95% Confidence Interval",
]
# %%
publication_table.columns = [
"Label Expand",
"CV-folds",
"Mean $\pm$ Std. Dev.",
"95% Confidence Interval",
]
publication_table = publication_table.iloc[:, 1:3]
publication_table
# %%
# Reporting validation set metrics
publication_table.to_latex()
# %%
# Just get the median value
f1_scores.groupby("explabel")[cur_feature_name].agg(
mean_val="median",
std_val="std",
ci_bounds=get_ci_bounds, # Uses the single function to get a tuple of bounds
count="size",
).reset_index()
# %% [markdown]
# ## 2.2 Test set - Plot confusion matrix with CI on the test set using best model only
# %%
# Evaluate all best model on the validation set
num_folds = 5
exp_labels = [2]
data_set = "test"
for i in range(num_folds):
for j in exp_labels:
# curr_model_path = np.sort(glob(f"/home/geromevivar/projects/gait_ml/backpain/Fold{i+1}*expandlabel{j}*/*/*"))[-1]
curr_model_path = np.sort(
glob(
f"/home/qivy00li/projects/gait_ml/backpain/ZscaledRerunExp4-Fold{i + 1}*expandlabel{j}*/*/*"
)
)[-1]
save_name = f"Fold{i + 1}_explabel{j}_{Path(curr_model_path).stem}.npz"
if data_set == "test":
save_name = f"{data_set}set_{save_name}"
save_name = save_name.replace(
"testset_", "rerunWithPredAndTargetSaved_testset_"
)
save_name = save_name.replace(".npz", ".pkl")
testset_results = eval(
data_set=data_set,
model_fpath=curr_model_path,
fold=i,
return_preds_targets=return_preds_targets,
)
if not os.path.exists(save_name):
print(f"=== Running model: {save_name} ===")
return_preds_targets = False
testset_results = eval(
data_set=data_set,
model_fpath=curr_model_path,
fold=i,
return_preds_targets=return_preds_targets,
)
if return_preds_targets:
with open(save_name, "wb") as f:
pickle.dump(testset_results, f)
else:
np.savez(
save_name, **testset_results
) # used in original paper experiments
# %%
testset_all_results = []
print(f"Currently evaluating expand label: {j}")
testset_fnames = np.sort(glob(f"testset_Fold*_explabel{j}*"))
for fname in testset_fnames:
print(f"Loading {fname}")
curr_results = np.load(fname, allow_pickle=True)
curr_results = {**curr_results}
curr_results["fold"] = i + 1
curr_results["explabel"] = j
testset_all_results.append(curr_results)
# %%
testset_all_results_df = pd.DataFrame(testset_all_results)
testset_all_results_df
# %%
testset_all_results_df.testset_mae.apply(lambda x: len(x))
# %%
cm_per_fold = testset_all_results_df.testset_cm.apply(
lambda x: np.stack(x).sum(0) / np.stack(x).sum(0).sum(1)
)
cm_per_fold
# %%
combined_cm = testset_all_results_df.testset_cm.apply(lambda x: np.stack(x).sum(0)).sum(
0
)
normalized_cm = (combined_cm / combined_cm.sum(1)).round(2)
# %%
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import confusion_matrix
# ... other imports ...
# def plot_confusion_matrix(cm, class_names, title, fmt, cbar_label, cmap=plt.cm.Blues, file_name='confusion_matrix.png'):
# # ... (function body as executed above) ...
# fig, ax = plt.subplots(figsize=(5, 5))
# sns.heatmap(
# cm,
# annot=True,
# fmt=fmt,
# cmap=cmap,
# linewidths=0.5,
# linecolor='black',
# cbar=False,
# # cbar_kws={'label': cbar_label, 'orientation': 'vertical', 'pad': 0.04, 'aspect': 30},
# annot_kws={"fontsize": 16, "fontweight": "bold"},
# ax=ax,
# square=True
# )
# ax.set_title(title, fontsize=12, fontweight='bold', pad=5)
# ax.set_ylabel('True Label', fontsize=20, fontweight='medium')
# ax.set_xlabel('Predicted Label', fontsize=20, fontweight='medium')
# # Set class labels on ticks, centering them
# tick_marks = np.arange(len(class_names))
# ax.set_xticks(tick_marks + 0.5)
# ax.set_yticks(tick_marks + 0.5)
# ax.set_xticklabels(class_names, fontsize=16)
# ax.set_yticklabels(class_names, fontsize=16, rotation=90, va="center")
# # Fix for half-pixel issues in matplotlib 3.1.1+ (sets the limits correctly)
# ax.set_ylim(len(class_names), 0)
# ax.tick_params(axis='both', which='major', length=0)
# plt.tight_layout()
# # ... (code to set ticks and save figure) ...
# # plt.savefig(file_name, dpi=300, bbox_inches='tight')
# # fig.show()
# # plt.close(fig)
# return fig
# def plot_confusion_matrix(cm, class_names, title=None, fmt='d', cbar_label='Count', cmap=plt.cm.Blues, file_name='confusion_matrix.pdf'):
# """
# Plots a publication-ready confusion matrix for JNER.
# """
# # 1. Set JNER-compliant font (Arial/Helvetica is standard)
# plt.rcParams['font.family'] = 'sans-serif'
# plt.rcParams['font.sans-serif'] = ['Arial', 'Helvetica', 'DejaVu Sans']
# # 2. Size: 3.5 inches is standard for single-column width (85-90mm)
# fig, ax = plt.subplots(figsize=(5, 5))
# sns.heatmap(
# cm,
# annot=True,
# fmt=fmt,
# cmap=cmap,
# linewidths=1.0, # Thicker lines for better separation in print
# linecolor='black',
# cbar=False, # Disable colorbar if numbers are annotated (saves space)
# annot_kws={"fontsize": 18, "fontweight": "bold"}, # Large font for readability when resized
# ax=ax,
# square=True
# )
# # 3. Titles: JNER prefers titles in the caption, not the image.
# # Only set if strictly necessary for internal use.
# if title:
# ax.set_title(title, fontsize=14, fontweight='bold', pad=10)
# # 4. Axis Labels: Clear and large
# ax.set_ylabel('True Event', fontsize=18, fontweight='bold')
# ax.set_xlabel('Predicted Event', fontsize=18, fontweight='bold')
# # 5. Ticks: Center them and ensure readability
# tick_marks = np.arange(len(class_names))
# ax.set_xticks(tick_marks + 0.5)
# ax.set_yticks(tick_marks + 0.5)
# ax.set_xticklabels(class_names, fontsize=16, fontweight='medium')
# # CHANGED: Rotation 0 is better for short labels like "IC/FO"
# ax.set_yticklabels(class_names, fontsize=16, fontweight='medium', rotation=0, va="center")
# # Cleanups
# ax.tick_params(axis='both', which='major', length=0)
# plt.tight_layout()
# # 6. Saving: Use 600 DPI for raster or PDF/EPS for vector (Best for JNER)
# # If saving as PNG, use 600 dpi. If PDF, dpi is less critical but good practice.
# plt.savefig(file_name, dpi=600, bbox_inches='tight', transparent=False)
# # plt.close(fig) # Uncomment to prevent display in notebooks if generating many
# return fig
# %%
from gait_ml import utils
for k, v in testset_all_results_df.iterrows():
print(k)
# plot_confusion_matrix((np.stack(v.testset_cm).sum(0)/np.stack(v.testset_cm).sum(0).sum(1)).round(2), ["NE", "IC", "TO"], "Detection Performance on Test Set", ".2f", ["NE", "IC", "TO"])
utils.plot_confusion_matrix(
cm=(np.stack(v.testset_cm).sum(0) / np.stack(v.testset_cm).sum(0).sum(1)).round(
2
),
class_names=["NE", "IC", "TO"],
title=None,
fmt=".2f",
cbar_label="Count",
file_name=f"X_figures/cm-f{k + 1}.pdf",
)
# %%
utils.plot_confusion_matrix(
cm=normalized_cm,
class_names=["NE", "IC", "TO"],
title=None,
fmt=".2f",
cbar_label="Count",
file_name=f"X_figures/aggregated_cm_testset.pdf",
)
# %% [markdown]
# ## 2.3 Table detection and MAE metrics on test set mean [std]
# - F1-score, Precision, Recall, TP, FP, FN, MAE per class per group
# %%
metrics_per_fold = []
for k, v in testset_all_results_df.iterrows():
print(f"==Processing fold: {k}==")
for j in range(2):
print(f"Group: {j}")
curr_metrics = dict()
curr_group = v.group
curr_metrics["f1-score"] = v.testset_pc_f1_scores[curr_group == j].mean(0)
curr_metrics["recall"] = v.testset_pc_recall[curr_group == j].mean(0)
curr_metrics["precision"] = v.testset_pc_precision[curr_group == j].mean(0)
curr_mae = pd.Series(v.testset_pc_mae[curr_group == j]).apply(
lambda x: [x[idx + 1] for idx in range(2)]
)
curr_mae = np.stack(curr_mae).mean(0)
curr_mae = np.hstack([np.array([np.nan]), curr_mae])
curr_metrics["mae"] = curr_mae * 10.0
curr_metrics["group"] = j
curr_metrics["fold"] = k + 1
metrics_per_fold.append(curr_metrics)
results_df = pd.DataFrame(metrics_per_fold)
# results_df.drop(columns=["fold"], inplace=True)
# %%
# Prepare dataframe for plotting
plot_df = results_df.melt(["group", "fold"])
event_df = pd.DataFrame(plot_df["value"].apply(pd.Series))
event_df.columns = ["NE", "IC", "FO"]
plot_df.drop(columns="value", inplace=True)
plot_df = pd.concat([plot_df, event_df], axis=1)
df_long = plot_df.melt(
id_vars=["group", "fold", "variable"],
value_vars=["NE", "IC", "FO"],
var_name="event", # New column for the variable names
value_name="value",
)
df_long.group.replace(0, "Healthy", inplace=True)
df_long.group.replace(1, "BackPain", inplace=True)
df_long.variable.replace("mae", "MAE", inplace=True)
df_long.rename(columns={"variable": "Metric", "group": "Group"}, inplace=True)
# %%
df_long.groupby(["Metric", "Group", "event"])["value"].describe().round(3)
# %%
import matplotlib.pyplot as plt
import seaborn as sns
# Assuming df_long is defined and contains your data
sns.set_context("poster")
print("+++++ WarningL: Excluding NE events in the plot!++++")
df_long = df_long[df_long.event != "NE"]
g = sns.catplot(
data=df_long,
x="event",
y="value",
hue="Group",
kind="box",
col="Metric",
col_wrap=2,
height=6,
aspect=1.2,
sharey=False,
sharex=False,
palette="colorblind",
)
# --- Step 1: Define the custom Titles, Y-labels, and Performance Goal ---
metrics = df_long["Metric"].unique()
# # Define the custom info for each metric, now including a custom title
# custom_metrics_info = {
# metrics[0]: {"title": "F1-Score Detection Performance", "label": "F1-Score (%)", "goal": "Higher is Better (↑)", "y_pos": 0.98},
# metrics[1]: {"title": "Mean Error Results", "label": "Mean Error [ms]", "goal": "Lower is Better (↓)", "y_pos": 0.05},
# metrics[2]: {"title": "Event Precision Analysis", "label": "Detection Precision", "goal": "Higher is Better (↑)", "y_pos": 0.98},
# metrics[3]: {"title": "Root Mean Squared Error [ms]", "label": "Root Mean Squared Error", "goal": "Lower is Better (↓)", "y_pos": 0.05}
# }
custom_metrics_info = {
metrics[0]: {"title": "F1-Score (↑ better)"},
metrics[1]: {"title": "Recall (↑ better)"},
metrics[2]: {"title": "Precision(↑ better)"},
metrics[3]: {"title": "Mean Absolute Error [ms] (↓ better)"},
}
# --- Step 2: Iterate and Apply Titles, Labels, and Annotations ---
for ax_index, ax in enumerate(g.axes.flat):
current_metric = metrics[ax_index]
info = custom_metrics_info.get(current_metric)
if info:
# ⭐ Key Customization 1: Set the custom plot title
ax.set_title(info["title"], fontsize=20, fontweight="bold")
# Set the custom Y-axis label
# ax.set_ylabel(info["label"], fontsize=18)
# # Add text annotation to indicate the goal
# ax.text(
# x=0.05,
# y=info["y_pos"],
# s=info["goal"],
# transform=ax.transAxes,
# fontsize=16,
# color='red' if 'Lower' in info["goal"] else 'green',
# fontweight='bold'
# )
# --- Step 3: Clean up shared labels and titles ---
# Remove the default shared label from the grid
g.set_axis_labels("Gait Events", "Values", fontweight="bold")
# Remove the default top-level title that catplot tries to set for the column
# g.set_titles(col_template='{col_name}', row_template='{row_name}', size=0) # Set size=0 to hide
# Add a main title to the figure (applies to the entire figure, not individual plots)
# g.fig.suptitle('Gait Event Detection Performance on Testset (5-fold CV)', y=1.03, fontsize=22)
# Improve tick labels (numbers/text on axes)
g.tick_params(axis="both", which="major", labelsize=20)
plt.savefig("X_figures/overall-performance2x2.pdf", bbox_inches="tight")
plt.savefig("X_figures/overall-performance2x2.png", dpi=600, bbox_inches="tight")
plt.show()
# %%
# plt.figure(figsize=(10, 5))
# g = sns.catplot(data=df_long,
# x="event",
# y="value",
# hue="Group",
# kind="box",
# col='Metric',
# col_wrap=2,
# height=6,
# aspect=1.2,
# sharey=False,
# palette='colorblind')
# # Add a main title to the figure
# g.fig.suptitle('Gait Event Detection Performance on Testset (5-fold CV)', y=1.03, fontsize=22)
# # Improve axis labels|
# g.tick_params(axis='both', which='major', labelsize=16)
# # plt.tight_layout(rect=[0, 0, 1, 0.97])
# g.set_axis_labels("Event", "Value")
# plt.show()
# %%
# # Create the plot
# g = sns.catplot(data=df_long,
# x="event",
# y="value",
# hue="Group",
# kind="box",
# col='Metric',
# # col_wrap=2,
# height=6,
# aspect=1.2,
# sharey=False,
# legend_out=True,
# palette='colorblind')
# # 1. Main Title
# # g.fig.suptitle('Gait Event Detection Performance on Testset (5-fold CV)', y=1.03, fontsize=20)
# # 2. Subplot Titles (e.g., "Metric = Accuracy")
# g.set_titles(size=25)
# # 3. Axis Labels
# g.set_axis_labels("Event", "Value", fontsize=25)
# # 4. Tick Labels
# g.tick_params(axis='both', which='major', labelsize=25)
# # 5. Legend Title and Labels
# if g.legend:
# plt.setp(g.legend.get_texts(), fontsize='25')
# plt.setp(g.legend.get_title(), fontsize='25')
# # Adjust layout
# # plt.tight_layout(rect=[0, 0, 1, 0.97])
# plt.show()
# %% [markdown]
# ### 3. Concordance Analysis
# %%
from gait_ml import utils
import numpy as np
# %%
# Evaluate all best model on the validation set
num_folds = 5
exp_labels = [2]
data_set = "test"
for i in range(num_folds):
if i == 0:
for j in exp_labels:
# curr_model_path = np.sort(glob(f"/home/geromevivar/projects/gait_ml/backpain/Fold{i+1}*expandlabel{j}*/*/*"))[-1]
curr_model_path = np.sort(
glob(
f"/home/qivy00li/projects/gait_ml/backpain/ZscaledRerunExp4-Fold{i + 1}*expandlabel{j}*/*/*"
)
)[-1]
save_name = f"Fold{i + 1}_explabel{j}_{Path(curr_model_path).stem}.npz"
if data_set == "test":
save_name = f"{data_set}set_{save_name}"
# if not os.path.exists(save_name):
print(f"=== Running model: {save_name} ===")
testset_results = eval(
data_set=data_set,
model_fpath=curr_model_path,
fold=i,
return_preds_targets=True,
)
# np.savez(save_name, **testset_results)
break
# %%
all_preds = testset_results["all_preds"]
all_targets = testset_results["all_targets"]
# %%
# Find all patient ids
# subject_ids = [Path(i).parents[3].name for i in test_set_fpaths]
# Take average feature per subject since there are ~
stridetime_pred = [utils.calculate_stride_times(i, 100) for i in all_preds]
stridetime_target = [utils.calculate_stride_times(i, 100) for i in all_targets]
stridetime_pred = pd.DataFrame([i.mean() for i in stridetime_pred])
stridetime_target = pd.DataFrame([i.mean() for i in stridetime_target])
# stance_pred = utils.aggregate_res(stance_pred, subject_ids)
# stance_target = utils.aggregate_res(stance_target, subject_ids)
stridetime_res_df = pd.concat([stridetime_pred, stridetime_target], axis=1)
stridetime_res_df
# %%
# Find all patient ids
# subject_ids = [Path(i).parents[3].name for i in test_set_fpaths]
# Take average feature per subject since there are ~
stance_pred = [utils.calculate_gait_phases_vectorized(i) for i in all_preds]
stance_target = [utils.calculate_gait_phases_vectorized(i) for i in all_targets]
stance_pred = pd.DataFrame([i.mean() for i in stance_pred])
stance_target = pd.DataFrame([i.mean() for i in stance_target])
# stance_pred = utils.aggregate_res(stance_pred, subject_ids)
# stance_target = utils.aggregate_res(stance_target, subject_ids)
# %%
res_df = pd.concat([stance_pred, stance_target], axis=1)
res_df
# %%
swing_prop_df = 100 - res_df
# %%
# 2. Call the plotting function
fig, ax = utils.plot_bland_altman_publication(
swing_prop_df.iloc[:, 0],
swing_prop_df.iloc[:, 1],
method1_name="Pred",
method2_name="GT",
units=r"[$\%_{Gait}$]",
filename="mdpi_figures/swing_bland_altman_internal.pdf",
feature_name="Swing Phase",
ypos=0.75,
)
# %%
# 2. Call the plotting function
fig, ax = utils.plot_bland_altman_publication(
res_df.iloc[:, 0],
res_df.iloc[:, 1],
method1_name="Pred",
method2_name="GT",
units=r"[$\%_{Gait}$]",
filename="mdpi_figures/stance_bland_altman_internal.pdf",
feature_name="Stance Phase",
)
# %%
# 2. Call the plotting function
fig, ax = utils.plot_bland_altman_publicationv2(
stridetime_res_df.iloc[:, 0],
stridetime_res_df.iloc[:, 1],
method1_name="Pred",
method2_name="GT",
units=r"ms",
filename="mdpi_figures/stridetime_bland_altman_internal.pdf",
feature_name="Stride Time",
)
# %% [markdown]
# ### Confidence intervals
# %%
ccc, ccc_ci_lower, ccc_ci_upper = utils.lins_ccc_with_ci(
stridetime_res_df.iloc[:, 0],
stridetime_res_df.iloc[:, 1],
confidence_level=0.95,
n_resamples=1000,
random_seed=42,
)
# %%
print("ccc:", round(ccc, 3))
print("ccc_ci_lower:", round(ccc_ci_lower, 3))
print("ccc_ci_upper:", round(ccc_ci_upper, 3))
# %%
ccc, ccc_ci_lower, ccc_ci_upper = utils.lins_ccc_with_ci(
res_df.iloc[:, 0],
res_df.iloc[:, 1],
confidence_level=0.95,
n_resamples=1000,
random_seed=42,
)
# %%
print("ccc:", round(ccc, 3))
print("ccc_ci_lower:", round(ccc_ci_lower, 3))
print("ccc_ci_upper:", round(ccc_ci_upper, 3))
# %%

03_RNN_eval.ipynb at commit 3c1aed0, under other · at the source

Overview

Authors: Gerome Vivar¹, Shivam Singh¹, Tobias Bea¹, Christian Saal¹, Victor Munoz-Martel¹, Lutz Schega¹

ORCID iDs: Gerome Vivar, Shivam Singh, Tobias Bea, Victor Munoz-Martel, Lutz Schega

Health and Physical Activity, Otto von Guericke University Magdeburg, 39104 Magdeburg, Germany; (S.S.); (L.S.)

Institutions: Otto-von-Guericke-Universität Magdeburg (Germany)

Journal: Bioengineering (Basel, Switzerland), volume 13, issue 8, article 924

Dates: received 16 July 2026; accepted 11 August 2026; published online 14 August 2026

Type: Research article · Language: English

License: CC BY

Identifiers: DOI 10.3390/bioengineering13080924 · PMID 42649812 · PMCID PMC13509232 · OpenAlex W7202368435

Open access: gold, a free copy (OpenAlex)

Status: code verified

Categories: other (modality), human (organism), pain (population), methods / tools (subfield)

Methods: Spectral & time-frequency, Connectivity, Machine learning, Preprocessing, Statistics

Keywords: deep learning, gait analysis, gait event detection, non-specific low back pain, smartphone IMU, transfer learning

Topic: Balance, Gait, and Falls Prevention (Physical Therapy, Sports Therapy and Rehabilitation, Health Professions), according to OpenAlex

Funding: European Regional Development Fund (ZS/2023/11/181945)

Citations: not cited yet (Europe PMC); 34 references in the paper

Abstract

Accurate gait event detection using inertial measurement units (IMUs) is essential for temporal gait analysis, but frame-level detection is challenged by sparse initial contact (IC) and foot-off (FO) events. This study evaluated recurrent neural network architectures and training strategies for simultaneous IC and FO detection using a single shank-mounted smartphone IMU. The internal dataset included 28 healthy older adults and 18 individuals with non-specific low back pain (NSLBP). Temporal label expansion substantially improved validation performance for gated recurrent unit (GRU) and long short-term memory models, whereas point-label and class-weighted training performed poorly. The selected label-expanded GRU (LE-GRU) achieved F1 scores above 0.95 for both events and mean absolute temporal errors below 12 ms on held-out internal test folds, with high performance in both cohorts. On an external dataset with different sensor and acquisition characteristics, high performance required full-network fine-tuning, indicating the need for adaptation across datasets. Stance phase and stride time calculated from LE-GRU-predicted events showed high agreement with reference-derived values, with Lin’s concordance correlation coefficients from 0.980 to 0.994. These findings demonstrate that temporal label expansion enables accurate GRU-based gait event detection and temporal gait analysis from data collected with a single smartphone IMU.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 4 matches between paragraphs and lines of code.

MS-AI-OVGU/gait_ml

License: other

State: the link answers, verified on 27 September 2026

Evidence: files inventoried

Commit: 3c1aed01345c63d395565a5ae21465c877d9591e, 27 August 2026

Languages: Python (13), Jupyter (9)

Size: 38 files, 22 scripts

Software Heritage: not archived

Found in: “Data Availability Statement”

Holds: README, license file, environment (pyproject.toml), 9 notebooks

Not found: CITATION.cff, tests, continuous integration, documentation

Tools: NumPy (8 files), pandas (8 files), PyTorch Lightning (7 files), PyTorch (7 files), Matplotlib (6 files), scikit-learn (6 files), SciPy (6 files), Plotly (4 files), seaborn (3 files)

Availability: 1 check, the latest on 27 September 2026: the link answers

27 September 2026: the link answers

10 files

notebooks/03_RNN-Copy1.ipynb, Jupyter, 545 lines, 1 match
notebooks/03_RNN-Copy2.ipynb, Jupyter, 563 lines, 1 match
notebooks/03_RNN.ipynb, Jupyter, 538 lines
notebooks/03_RNN_eval.ipynb, Jupyter, 1,600 lines, 2 matches
notebooks/03_RNN_eval_external.ipynb, Jupyter, 345 lines
notebooks/03_RNN_finetuning.ipynb, Jupyter, 1,341 lines
notebooks/04_demographics_table.ipynb, Jupyter, 135 lines
notebooks/data_checks.ipynb, Jupyter, 149 lines
repository limit reached (2,000 files or 30 MB): the rest is at the source (14 files)
LICENSE, License, 133 lines
README.md, Text, 54 lines

At the source: github.com/MS-AI-OVGU/gait_ml

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
8 scripts, each with its path and the digest of its content;
4 matches between paragraphs of the paper and lines of the code (method lexical-v1);
neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

zenodo:17899477, at Zenodo; found in “Data Availability Statement”

Data Availability Statement

The internal data supporting the findings of this study is deposited in Zenodo and available at https://doi.org/10.5281/zenodo.17899477 (accessed 10 August 2026). The source code is available at https://github.com/MS-AI-OVGU/gait_ml (accessed 10 August 2026). The analyses were performed using Python 3.10.18, and the repository includes the package dependencies and version information required to reproduce the analyses. The source code is released under the PolyForm Noncommercial License 1.0.0 for noncommercial research use. The external dataset is publicly available as described in [28].

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 6 authors, 6 keywords, 1 funder, 30 references.

Cite

This paper

Vivar, G., Singh, S., Bea, T., Saal, C., Munoz-Martel, V., & Schega, L. (2026). Deep Learning-Based Temporal Gait Analysis Using a Smartphone IMU in Older Adults with and Without Non-Specific Low Back Pain. Bioengineering (Basel, Switzerland), 13(8), 924. https://doi.org/10.3390/bioengineering13080924

BibTeX

@article{vivar2026deep, author = {Vivar, Gerome and Singh, Shivam and Bea, Tobias and Saal, Christian and Munoz-Martel, Victor and Schega, Lutz}, title = {{Deep Learning-Based Temporal Gait Analysis Using a Smartphone IMU in Older Adults with and Without Non-Specific Low Back Pain}}, journal = {Bioengineering (Basel, Switzerland)}, year = {2026}, month = aug, volume = {13}, number = {8}, pages = {924}, publisher = {Multidisciplinary Digital Publishing Institute (MDPI)}, issn = {2306-5354}, doi = {10.3390/bioengineering13080924}, url = {https://doi.org/10.3390/bioengineering13080924}, pmid = {42649812}, pmcid = {PMC13509232} }

RIS

TY - JOUR AU - Vivar, Gerome AU - Singh, Shivam AU - Bea, Tobias AU - Saal, Christian AU - Munoz-Martel, Victor AU - Schega, Lutz TI - Deep Learning-Based Temporal Gait Analysis Using a Smartphone IMU in Older Adults with and Without Non-Specific Low Back Pain T2 - Bioengineering (Basel, Switzerland) J2 - Bioengineering (Basel) PY - 2026 DA - 2026/08/14 VL - 13 IS - 8 SP - 924 SN - 2306-5354 PB - Multidisciplinary Digital Publishing Institute (MDPI) DO - 10.3390/bioengineering13080924 UR - https://doi.org/10.3390/bioengineering13080924 LA - en ER -

CSL-JSON

{ "id": "10.3390/bioengineering13080924", "type": "article-journal", "title": "Deep Learning-Based Temporal Gait Analysis Using a Smartphone IMU in Older Adults with and Without Non-Specific Low Back Pain", "container-title": "Bioengineering (Basel, Switzerland)", "author": [ { "family": "Vivar", "given": "Gerome" }, { "family": "Singh", "given": "Shivam" }, { "family": "Bea", "given": "Tobias" }, { "family": "Saal", "given": "Christian" }, { "family": "Munoz-Martel", "given": "Victor" }, { "family": "Schega", "given": "Lutz" } ], "container-title-short": "Bioengineering (Basel)", "volume": "13", "issue": "8", "page": "924", "DOI": "10.3390/bioengineering13080924", "PMID": "42649812", "PMCID": "PMC13509232", "ISSN": "2306-5354", "publisher": "Multidisciplinary Digital Publishing Institute (MDPI)", "URL": "https://doi.org/10.3390/bioengineering13080924", "language": "en", "issued": { "date-parts": [ [ 2026, 8, 14 ] ] } }

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.