Domain adaptive uplift modeling across heterogeneous mental health cohorts.

Code ↔ Paper

10 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 10 matches

[1] § STAR★Methods › Method details › Datasets and problem setup ↔ dann-analysis.ipynb, lines 97–116 · score 0.80 · medical student, academic stress, habit change, general population, EEG, cognitive
[2] § STAR★Methods › Method details › Construct validity and comparability › Interpretation caveat ↔ dann-analysis.ipynb, lines 169–203 · score 0.78 · main.disorder, specific.disorder, mood disorder, treatment proxy, outcome proxy, anxiety
[3] § STAR★Methods › Method details › Notation and formal definitions › Domain-adversarial training ↔ dann-analysis.ipynb, lines 454–594 · score 0.72 · gradient reversal layer, domain discriminator, optimized, scalar, GRL, loss
[4] § STAR★Methods › Method details › Construct validity and comparability › Measurement heterogeneity ↔ dann-analysis.ipynb, lines 97–116 · score 0.71 · medical student, burnout exhaustion, clinical diagnosis, growing stress, treatment proxy, populations
[5] § STAR★Methods › Method details › Model: DANN_Uplift (domain-adversarial uplift network) › Adversarial domain alignment and fairness regularizer ↔ dann-analysis.ipynb, lines 454–594 · score 0.71 · cross entropy, domain loss, masks, discriminator, GRL, binary
[6] § Results › Systematic λD analysis and diagnostics › Fairness diagnostics ↔ dann-analysis.ipynb, lines 169–203 · score 0.69 · main.disorder, specific.disorder, mood disorder, treatment proxy, outcome proxy, anxiety
[7] § Results › Head-to-head: DANN vs. CDAN vs. CORAL vs. MMD vs. NoDA ↔ dann-analysis.ipynb, lines 889–1031 · score 0.65 · Wilcoxon signed rank, paired median, histogram, CI, NoDA, Cliff
[8] § STAR★Methods › Method details › Evaluation protocol ↔ dann-analysis.ipynb, lines 889–1031 · score 0.56 · Wilcoxon signed rank, nonparametric, Cliff, bootstrap, gaps, Fairness
[9] § Results › Schedules, transfers, and semi-synthetic validation › Scheduled λD ↔ dann-analysis.ipynb, lines 1290–1324 · score 0.53 · linear ramp, sigmoid ramp, schedules, DANN, AUUC
[10] § STAR★Methods › Method details › Notation and formal definitions › Area under the uplift curve (AUUC) ↔ dann-analysis.ipynb, lines 1191–1238 · score 0.53 · uplift curve, random baseline, fractions, bins, inverted, ranking

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The paper is loaded by your browser, with JavaScript: read it at doi.org or on Europe PMC.

The authors' code

Jupyter notebook · 2,286 lines · 86 KB · no license · 10 matches

# %%
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory
import os
for dirname, _, filenames in os.walk('/kaggle/input'):
for filename in filenames:
print(os.path.join(dirname, filename))
# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All"
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
# %% [markdown]
# ### This code integrates Domain-Adversarial Neural Networks (DANN) into a cross-dataset mental health study by harmonizing and processing data from three distinct sources: ###
#
#
# * **Medical Students' Mental Health (med)**
# * **Mental Health During Quarantine (quar)**
# * **Psychiatric Disorders Dataset (psych)**
#
#
#
# ### DANN is used to learn shared mental health patterns while adapting to domain-specific differences across populations. ###
# %%
from pathlib import Path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
from IPython.display import display
import seaborn as sns
import re
import random
import statsmodels.api as sm
from statsmodels.formula.api import ols
from scipy.stats import linregress
from scipy.stats import ttest_ind, chi2_contingency
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from scipy import sparse
import torch, torch.nn as nn, torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader, TensorDataset
from pathlib import Path
from sklearn.compose import ColumnTransformer
import pytorch_lightning as pl
import joblib
from sklearn.model_selection import GroupKFold
from sklearn.metrics import roc_auc_score
import optuna, json, gc, os
import warnings; warnings.filterwarnings("ignore")
# %%
def clean_dataframe(df):
# Identify and drop completely empty unnamed columns
unnamed_cols = [col for col in df.columns if "unnamed" in col]
empty_unnamed_cols = [col for col in unnamed_cols if df[col].isnull().all()]
df.drop(empty_unnamed_cols, axis=1, inplace=True)
# Identify columns with NaN values
nan_counts = df.isna().sum()
columns_with_nan = nan_counts[nan_counts != 0].index.tolist()
print("Columns identified with NaNs:", columns_with_nan)
print("Rows affected before dropna:", df[columns_with_nan].isna().sum().sum())
# Drop rows where any of the identified columns have NaN values
if columns_with_nan:
df.dropna(subset=columns_with_nan, inplace=True)
print("Final dataset shape:", df.shape, "\n")
return df
# %%
def make_tensor_dataset(X, y, T, dom, gender, edu):
X = torch.tensor(X, dtype=torch.float32)
y = torch.tensor(y, dtype=torch.float32).unsqueeze(1)
T = torch.tensor(T, dtype=torch.float32).unsqueeze(1)
dom = torch.tensor(dom, dtype=torch.long)
g = torch.tensor(gender, dtype=torch.float32).unsqueeze(1)
e = torch.tensor(edu, dtype=torch.float32).unsqueeze(1)
return TensorDataset(X, T, y, dom, g, e)
# %% [markdown]
# **Quarantine Mental Health (General Population)** : Defining Treatment (T)= Growing Stress & Outcome (Y)= Coping Struggles. Keeps relevant psychological indicators (e.g., frustration, habit changes).
# These will help DANN learn stress patterns specific to quarantine conditions.
#
# **Medical Students' Mental Health (Academic Burnout)** : Treatment Proxy (T)= Burnout Exhaustion (mbi_ex) & Outcome Proxy (Y)= Depression. Includes burnout (MBI), anxiety (STAI), empathy (QCAE), and academic stress predictors. Helps DANN differentiate academic stress effects from psychiatric disorders.
#
# **Psychiatric Disorders (Clinical Diagnoses)** : Treatment Proxy (T)= Stress & Anxiety (Categorizes patients diagnosed with anxiety disorders as T = 1) & Outcome Proxy (Y)= Depression (Categorizes patients with mood disorders & major depression as Y = 1). IQ is retained as a predictor of cognitive impairment across domains.
# %%
# ================================================================
# 0 PATHS
# ================================================================
ROOT = Path("/kaggle/input/datasets/anwe9274/three-datasets")
FILES = {
"med" : ROOT / "Medical Student Mental Health.csv",
"quar" : ROOT / "mental_health_data.csv",
"psych": ROOT / "EEG.machinelearing_data_BRMH.csv",
}
# ================================================================
# 1 Load & lower-case column names
# ================================================================
dfs = {k: pd.read_csv(p).rename(str.lower, axis=1) for k, p in FILES.items()}
# helpers ---------------------------------------------------
def pct75(s: pd.Series) -> float:
return s.quantile(0.75, interpolation="midpoint")
def _yes(s): # yes/no/maybe → 1/0 mapping
return s.astype(str).str.strip().str.lower().map({"yes": 1, "maybe": 1, "no": 0})
# ================================================================
# 2 Dataset-specific wrangling → returns df with unified columns
# ================================================================
def prep_quarantine(df: pd.DataFrame) -> pd.DataFrame:
df = clean_dataframe(df)
out = pd.DataFrame()
out["domain"] = "quar"
out["age"] = pd.to_numeric(df["age"], errors="coerce")
out["gender"] = df["gender"].str.strip().str.title().map({"Male": 1, "Female": 0})
out["edu_years"] = np.nan # not provided
# Treatment / outcome
out["T"] = _yes(df["growing_stress"])
out["Y"] = _yes(df["coping_struggles"])
# raw predictors (keep everything except obvious identifiers)
keep = ["occupation","days_indoors","quarantine_frustrations",
"changes_habits","mental_health_history","weight_change",
"mood_swings","work_interest","social_weakness"]
out = pd.concat([out, df[keep]], axis=1)
return out
def prep_medical(df: pd.DataFrame) -> pd.DataFrame:
df = clean_dataframe(df)
df = df[df["sex"] != 3]
out = pd.DataFrame()
out["domain"] = "med"
out["age"] = df["age"]
out["gender"] = df["sex"].map({1:1,2:0}) # 1=M,2=F in paper
out["edu_years"] = 12 + df["year"] # MBBS year 1-6
# Treatment / outcome
out["T"] = (df["mbi_ex"] >= pct75(df["mbi_ex"])).astype(int)
out["Y"] = (df["cesd"] >= 10).astype(int)
# predictors
predictors = ["stai_t","mbi_cy","mbi_ea","qcae_cog","qcae_aff",
"amsp","erec_mean","health","psyt","glang","stud_h"]
out = pd.concat([out, df[predictors]], axis=1)
return out
def prep_psych(df: pd.DataFrame) -> pd.DataFrame:
df = clean_dataframe(df)
out = pd.DataFrame()
# --- demographics -------------------------------------------
out["domain"] = "psych"
out["age"] = df["age"]
out["gender"] = df["sex"].str[0].str.upper().map({"M": 1, "F": 0})
out["edu_years"] = df["education"]
# --- categorical strings normalised -------------------------
main = df["main.disorder"].str.strip().str.lower()
specific = df["specific.disorder"].str.strip().str.lower()
# ---- treatment proxy (stress / anxiety) -------------------
stress_main = main.eq("anxiety disorder")
stress_spec = specific.isin(
{"posttraumatic stress disorder",
"panic disorder",
"social anxiety disorder",
}
)
out["T"] = (stress_main | stress_spec).astype(int)
# ---- outcome proxy (depressive condition) -----------------
dep_main = main.eq("mood disorder")
dep_spec = specific.eq("depressive disorder")
out["Y"] = (dep_main | dep_spec).astype(int)
# ---- extra predictors --------------------------------------
out["iq"] = df["iq"]
out["main_disorder"] = main # leave as feature
out["spec_disorder"] = specific
return out
# run preppers ----------------------------------------------------
dfs["quar"] = prep_quarantine(dfs["quar"])
dfs["med"] = prep_medical(dfs["med"])
dfs["psych"] = prep_psych(dfs["psych"])
label_map = {"med":0, "quar":1, "psych":2}
for k, df in dfs.items():
df["domain"] = label_map[k]
# ================================================================
# 3 Concatenate & basic cleaning
# ================================================================
df_all = pd.concat(dfs.values(), ignore_index=True)
domains = df_all["domain"].values.astype(np.int64) # <-- final label vector
# Education bin (≥12 yrs) – if NaN, set to population mode 0
df_all["edu_bin"] = (df_all["edu_years"].fillna(0) >= 12).astype(int)
# drop obvious ID cols that slipped through
drop_like = re.compile(r"id$|idx|record.*", re.I)
df_all = df_all[df_all.columns[~df_all.columns.str.match(drop_like)]]
# ================================================================
# 4 Pre-processing pipeline (numeric median / categorical mode)
# ================================================================
num_cols = df_all.select_dtypes(include=["number"]).columns.tolist()
cat_cols = [c for c in df_all.columns if c not in num_cols]
numeric_pipe = Pipeline([
("imp", SimpleImputer(strategy="median")),
])
categorical_pipe = Pipeline([
("imp", SimpleImputer(strategy="most_frequent")),
("ohe", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
])
ct = ColumnTransformer(
[("num", numeric_pipe, num_cols),
("cat", categorical_pipe, cat_cols)],
remainder="drop"
)
# ---------- ColumnTransformer ----------
X = ct.fit_transform(df_all) # shape (2629, 57)
y = df_all["Y"].values.astype(np.float32)
T = df_all["T"].values.astype(np.float32)
gender = df_all["gender"].values.astype(np.int64)
edu = df_all["edu_bin"].values.astype(np.int64)
# ---------- TensorDataset ----------
full_ds = make_tensor_dataset(X, y, T, domains, gender, edu)
print("Final feature matrix", X.shape, "with", len(num_cols), "numeric +",
len(ct.named_transformers_["cat"]["ohe"].get_feature_names_out()), "one-hot dims")
# save
joblib.dump(ct, "preproc_column_transformer.pkl")
df_all.to_csv("master_dataframe.csv", index=False)
# %%
for name, df in [("Medical", dfs["med"]),
("Quarantine", dfs["quar"]),
("Psychiatric", dfs["psych"])]:
print(f"{name}: n={len(df)} "
f"Prev(T)={df['T'].mean():.3f} "
f"Prev(Y)={df['Y'].mean():.3f}")
# %%
psych_raw = pd.read_csv(ROOT / "EEG.machinelearing_data_BRMH.csv")
psych_raw.columns = psych_raw.columns.str.lower()
unnamed_cols = [col for col in psych_raw.columns if "unnamed" in col]
empty_unnamed_cols = [col for col in unnamed_cols if psych_raw[col].isnull().all()]
psych_raw.drop(empty_unnamed_cols, axis=1, inplace=True)
nan_counts = psych_raw.isna().sum()
columns_with_nan = nan_counts[nan_counts != 0].index.tolist()
print("Columns identified with NaNs:", columns_with_nan)
print("Rows affected before dropna:", psych_raw[columns_with_nan].isna().sum().sum())
if columns_with_nan:
psych_raw.dropna(subset=columns_with_nan, inplace=True)
# --- Disorder proxies ---
main = psych_raw["main.disorder"].str.strip().str.lower()
specific = psych_raw["specific.disorder"].str.strip().str.lower()
stress_main = main.eq("anxiety disorder")
stress_spec = specific.isin({"posttraumatic stress disorder",
"panic disorder", "social anxiety disorder"})
T_psych = (stress_main | stress_spec).astype(int)
dep_main = main.eq("mood disorder")
dep_spec = specific.eq("depressive disorder")
Y_psych = (dep_main | dep_spec).astype(int)
# --- Diagnostics ---
ct = pd.crosstab(T_psych, Y_psych, margins=True,
rownames=["T (anxiety proxy)"],
colnames=["Y (mood/MDD proxy)"])
print(ct)
print()
print(f"P(Y=1 | T=1) = {Y_psych[T_psych==1].mean():.3f}")
print(f"P(Y=1 | T=0) = {Y_psych[T_psych==0].mean():.3f}")
# Overlap checks
both_from_main = main.eq("anxiety disorder") & main.eq("mood disorder")
print(f"Patients where main.disorder drives BOTH T and Y: {both_from_main.sum()}")
spec_T = specific.isin({"posttraumatic stress disorder",
"panic disorder", "social anxiety disorder"})
spec_Y = specific.eq("depressive disorder")
print(f"Patients with T=1 from specific AND Y=1 from specific: {(spec_T & spec_Y).sum()}")
print(f"Patients with T=1 from main AND Y=1 from specific: {(stress_main & dep_spec).sum()}")
print(f"Patients with T=1 from specific AND Y=1 from main: {(stress_spec & dep_main).sum()}")
# Feature inspection
print("\nFeatures retained from prep_psych that encode disorder:")
print("main_disorder unique values:", main.unique().tolist())
print("spec_disorder sample:", specific.value_counts().head(8).to_dict())
# %%
import torch
def check_tensor_dataset(dataset):
for i, tensor in enumerate(dataset.tensors):
nan_mask = torch.isnan(tensor)
inf_mask = torch.isinf(tensor)
if nan_mask.any():
print(f"NaNs found in tensor {i}: {nan_mask.nonzero(as_tuple=True)}")
if inf_mask.any():
print(f"Infinities found in tensor {i}: {inf_mask.nonzero(as_tuple=True)}")
print("Check complete.")
check_tensor_dataset(full_ds)
# %%
print(f"Dataset size: {len(full_ds)} samples")
for i in range(5): # Display first 5 samples
print(full_ds[i])
# %% [markdown]
# **0- Quarantine-->Fold 1, 1- Medical-->Fold 2, 3- Psychiatric-->Fold 3**
# %%
assert df_all["Y"].isin([0,1]).all(), "Found invalid Y values"
assert df_all["T"].isin([0,1]).all(), "Found invalid T values"
assert not np.isnan(X).any(), "NaNs found in X!"
assert np.isfinite(X).all(), "Inf found in X!"
print(df_all["domain"].value_counts(dropna=False))
# %%
domain_name = {0: "Medical", 1: "Quarantine", 2: "Psychiatric"}
gkf = GroupKFold(n_splits=3)
fold_domains = []
for fold, (tr, vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains), 1):
heldout = np.unique(domains[vl])
assert len(heldout) == 1
d = int(heldout[0])
fold_domains.append(d)
print(f"Fold {fold}: held-out domain {d} ({domain_name[d]})")
# %%
def auuc_score(y_true, treat, uplift_pred, n_bins=10):
# discretise uplift into percentile bins → cumulative net lift
df = (
pd.DataFrame({"y":y_true,"t":treat,"upl":uplift_pred})
.sort_values("upl", ascending=False)
.reset_index(drop=True)
)
step = int(np.ceil(len(df) / n_bins))
net_lift, random = [], []
for i in range(step, len(df)+1, step):
take = df.iloc[:i]
# uplift = E[y|T=1] - E[y|T=0]
t_on = take[take.t==1].y.mean() if (take.t==1).any() else 0
t_off = take[take.t==0].y.mean() if (take.t==0).any() else 0
net_lift.append(t_on - t_off)
random.append(df.iloc[:i].y.mean() - df.y.mean())
return np.trapz(net_lift) - np.trapz(random) # AUUC vs random baseline
def uplift_curve(y_true, treat, uplift_pred, bins=10):
df = pd.DataFrame({"y":y_true, "t":treat, "u":uplift_pred})\
.sort_values("u", ascending=False).reset_index(drop=True)
step = int(np.ceil(len(df)/bins))
xs, lift = [], []
for i in range(step, len(df)+1, step):
take = df.iloc[:i]
mu1 = take[take.t==1].y.mean() if (take.t==1).any() else 0
mu0 = take[take.t==0].y.mean() if (take.t==0).any() else 0
xs.append(i/len(df)); lift.append(mu1-mu0)
return xs, lift, float(np.trapz(lift))
def fairness_gap(uplift, mask_a, mask_b):
if mask_a.any() and mask_b.any():
return float(abs(uplift[mask_a].mean() - uplift[mask_b].mean()))
return 0.0
def make_loaders(tr_idx, vl_idx, bs=128):
return (
DataLoader(torch.utils.data.Subset(full_ds, tr_idx), bs, True, num_workers=2),
DataLoader(torch.utils.data.Subset(full_ds, vl_idx), bs*2,False, num_workers=2)
)
def eval_cfg(lr, λ_D, λ_F, max_ep=20):
gkf = GroupKFold(n_splits=3)
folds = gkf.split(np.zeros(len(y)), y, groups=domains)
auucs = []
for k,(tr,vl) in enumerate(folds,1):
mdl = DANNSystem(d_in=X.shape[1], λ_D=λ_D, λ_F=λ_F, lr=lr)
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep, accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, log_every_n_steps=5, enable_checkpointing=False,
gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc",mode="max",patience=5)],
)
trainer.fit(mdl, trL, vlL)
best = trainer.callback_metrics["val_auuc"].item()
print(f" fold{k}: best_AUUC={best:.3f}")
auucs.append(best)
del mdl; gc.collect(); torch.cuda.empty_cache()
mean,std = np.mean(auucs), np.std(auucs)
print(f"⇒ mean AUUC={mean:.3f} ±{std:.3f}\n")
return -mean, auucs # Optuna minimizes
# %% [markdown]
# Goal: estimate uplift (causal treatment effect) while
# (i) generalising across 3 data-collection domains and
# (ii) enforcing group-fairness in the predicted uplift.
#
# Encoder – turns raw tabular features x into a latent vector z.
# We want z to be useful for uplift but hide domain-specific quirks.
#
# Treatment heads – two sigmoid heads (head0, head1) predict the probability of a positive outcome under control and treatment respectively.
# Uplift = ŷ₁ − ŷ₀.
#
# Domain discriminator + GRL – a small classifier tries to recover the domain label (0, 1, 2) from z, but its gradient flows through a Gradient-Reversal Layer (GRL) which multiplies it by −λ<sub>D</sub>.
# Result: encoder is trained adversarially to make domains indistinguishable → better out-of-domain generalisation.
#
# Three loss terms
#
# Uplift loss L_up – standard binary cross-entropy on observed outcomes.
#
# Domain loss L_dom – cross-entropy of discriminator vs. true domain.
#
# Fairness loss L_fair – squared gap of average uplift between protected-group pairs (gender g and ethnicity e masks).
#
# Total loss = L_up + λ_D · L_dom + λ_F · L_fair.
#
# Validation metric – AUUC (Area Under Uplift Curve) logged each epoch.
# %% [markdown]
# # Encoder: 128-64 with Dropout 0.30
# %%
# --------------------------------------------
# 1. Grad-reversal helper
# --------------------------------------------
class GradReverse(torch.autograd.Function):
@staticmethod
def forward(ctx, x, l): ctx.l = l; return x.view_as(x)
@staticmethod
def backward(ctx, g): return -ctx.l * g, None
class GRL(nn.Module):
def __init__(self, l=1.0): super().__init__(); self.l=l
def forward(self, x): return GradReverse.apply(x, self.l)
# --------------------------------------------
# 2. Base model with optimiser + val logging
# --------------------------------------------
class DANN_Uplift(pl.LightningModule):
def __init__(self, d_in,
λ_D=1.0, λ_F=10.0,
lr=1e-4,
wd=1e-4):
super().__init__()
self.save_hyperparameters()
# —— Encoder: 128-64 with Dropout 0.3——
self.encoder = nn.Sequential(
nn.Linear(d_in, 128), nn.ReLU(), nn.Dropout(0.3),
nn.Linear(128, 64), nn.ReLU()
)
# —— Treatment heads ——
self.head0 = nn.Linear(64, 1)
self.head1 = nn.Linear(64, 1)
# —— Domain discriminator (3 classes) ——
self.disc = nn.Sequential(
nn.Linear(64, 64), nn.ReLU(),
nn.Linear(64, 3)
)
self.grl = GRL()
# -----------------------------------------------------------
def forward(self, x, t):
z = self.encoder(x)
y0 = torch.sigmoid(self.head0(z))
y1 = torch.sigmoid(self.head1(z))
d = self.disc(self.grl(z))
return y0, y1, d
# -----------------------------------------------------------
def training_step(self, batch, *_):
x, t, y, dom, g, e = batch
y0, y1, d_hat = self(x, t)
# ---------- primary & domain losses ----------
eps = 1e-7
y_pred = torch.clip(torch.where(t == 1, y1, y0), eps, 1 - eps)
L_up = F.binary_cross_entropy(y_pred, y)
L_dom = F.cross_entropy(d_hat, dom)
# ---------- fairness (safe) ----------
uplift = (y1 - y0).flatten()
def safe_gap(mask_a, mask_b):
if mask_a.any() and mask_b.any(): # both sub-groups present
return (uplift[mask_a].mean() - uplift[mask_b].mean()) ** 2
else:
return torch.scalar_tensor(0.0, device=x.device)
Δg = safe_gap(g.flatten() == 0, g.flatten() == 1)
Δe = safe_gap(e.flatten() == 0, e.flatten() == 1)
L_fair = Δg + Δe
# ---------- total loss ----------
loss = L_up + self.hparams.λ_D * L_dom + self.hparams.λ_F * L_fair
self.log_dict(
{"train_loss": loss,
"train_uplift": L_up,
"train_dom": L_dom,
"train_fair": L_fair},
prog_bar=True, on_epoch=True
)
return loss
# -----------------------------------------------------------
def validation_step(self, batch, *_):
x,t,y,dom,_,_ = batch
y0,y1,_ = self(x,t)
uplift_pred = (y1 - y0).detach().cpu().numpy().ravel()
uplift_true = y.detach().cpu().numpy().ravel()
treat = t.detach().cpu().numpy().ravel()
auuc = auuc_score(uplift_true, treat, uplift_pred)
self.log("val_auuc", auuc, prog_bar=True, on_epoch=True)
return {"auuc": auuc}
# -----------------------------------------------------------
def configure_optimizers(self):
opt = torch.optim.Adam(self.parameters(),
lr=self.hparams.lr,
weight_decay=self.hparams.wd)
return {"optimizer": opt, "gradient_clip_val": 1.0}
# --------------------------------------------
# 3. Thin wrapper
#
# --------------------------------------------
class DANNSystem(DANN_Uplift):
"""Inherits everything."""
pass
# %%
model = DANN_Uplift(d_in=100) # example input dimension
print(model)
# %%
def run_method(method_name, config, n_splits=3, max_ep=20):
gkf = GroupKFold(n_splits=n_splits)
auucs, gap_g, gap_e = [], [], []
for fold,(tr,vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains),1):
# Create model according to method
if method_name == "NoDA":
mdl = DANNSystem(d_in=X.shape[1], lr=config["lr"], λ_D=0.0, λ_F=config["λ_F"])
else: # "DANN" or others
mdl = DANNSystem(d_in=X.shape[1], lr=config["lr"], λ_D=config["λ_D"], λ_F=config["λ_F"])
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(max_epochs=max_ep, devices=1, accelerator="gpu"
if torch.cuda.is_available() else "cpu",
enable_checkpointing=False, gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc",mode="max",patience=5)],
logger=False)
trainer.fit(mdl, trL, vlL)
auuc = trainer.callback_metrics["val_auuc"].item(); auucs.append(auuc)
# compute uplift on validation set (same as your code)
u,tv,yv = [],[],[]
for xb,tb,yb,_,_,_ in vlL:
xb = xb.to(mdl.device)
with torch.no_grad():
z = mdl.encoder(xb)
u.append((mdl.head1(z)-mdl.head0(z)).cpu().numpy().ravel())
tv.append(tb.numpy().ravel()); yv.append(yb.numpy().ravel())
u = np.concatenate(u); tv=np.concatenate(tv); yv=np.concatenate(yv)
# fairness masks must be indexed by vl
xs,lift,_ = uplift_curve(yv,tv,u,bins=10)
gap_g.append(fairness_gap(u, (gender[vl]==0)&(tv==1), (gender[vl]==1)&(tv==1)))
gap_e.append(fairness_gap(u, (edu[vl]==0)&(tv==1), (edu[vl]==1)&(tv==1)))
del mdl; gc.collect(); torch.cuda.empty_cache()
return np.array(auucs), np.array(gap_g), np.array(gap_e)
# %%
def eval_uplift_and_gaps(mdl, vlL, vl_idx):
u, tv, yv = [], [], []
mdl.eval()
for xb, tb, yb, _, _, _ in vlL:
xb = xb.to(mdl.device)
with torch.no_grad():
z = mdl.encoder(xb)
u.append((mdl.head1(z) - mdl.head0(z)).cpu().numpy().ravel())
tv.append(tb.numpy().ravel()); yv.append(yb.numpy().ravel())
u = np.concatenate(u); tv = np.concatenate(tv); yv = np.concatenate(yv)
# uplift only on treated for fairness diagnostics (as in paper)
treated_mask = (tv == 1)
gap_g = fairness_gap(u, (gender[vl_idx] == 0) & treated_mask, (gender[vl_idx] == 1) & treated_mask)
gap_e = fairness_gap(u, (edu[vl_idx] == 0) & treated_mask, (edu[vl_idx] == 1) & treated_mask)
return u, tv, yv, gap_g, gap_e
def loso_with_gaps(tag, use_dann, max_ep=20):
gkf = GroupKFold(n_splits=3)
auucs, gap_gs, gap_es = [], [], []
for k,(tr,vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains),1):
mdl = build_model(use_dann=use_dann) # uses same defaults/hyperparams
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1,
enable_checkpointing=False,
callbacks=[pl.callbacks.EarlyStopping("val_auuc", mode="max", patience=5)],
logger=False, log_every_n_steps=5,
)
trainer.fit(mdl, trL, vlL)
auuc = trainer.callback_metrics["val_auuc"].item()
print(f"{tag} fold{k}: AUUC = {auuc:+.3f}")
# compute uplift and fairness gaps on this validation fold
_, tv, yv, gap_g, gap_e = eval_uplift_and_gaps(mdl, vlL, vl)
auucs.append(auuc); gap_gs.append(gap_g); gap_es.append(gap_e)
del mdl; gc.collect(); torch.cuda.empty_cache()
print(f"{tag} mean AUUC = {np.mean(auucs):+.3f}\n")
return np.array(auucs), np.array(gap_gs), np.array(gap_es)
# %%
# ======================================================================
# 0 · Reproducible seed
# ======================================================================
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
torch.cuda.manual_seed_all(SEED); torch.backends.cudnn.deterministic = True
optuna.logging.set_verbosity(optuna.logging.WARNING) # silence widget warnings
def uplift_curve(y_true, treat, uplift_pred, bins=10):
df = pd.DataFrame({"y":y_true, "t":treat, "u":uplift_pred})\
.sort_values("u", ascending=False).reset_index(drop=True)
step = int(np.ceil(len(df)/bins))
xs, lift = [], []
for i in range(step, len(df)+1, step):
take = df.iloc[:i]
mu1 = take[take.t==1].y.mean() if (take.t==1).any() else 0
mu0 = take[take.t==0].y.mean() if (take.t==0).any() else 0
xs.append(i/len(df)); lift.append(mu1-mu0)
return xs, lift, float(np.trapz(lift))
def fairness_gap(uplift, mask_a, mask_b):
if mask_a.any() and mask_b.any():
return float(abs(uplift[mask_a].mean() - uplift[mask_b].mean()))
return 0.0
def make_loaders(tr_idx, vl_idx, bs=128):
return (
DataLoader(torch.utils.data.Subset(full_ds, tr_idx), bs, True, num_workers=2),
DataLoader(torch.utils.data.Subset(full_ds, vl_idx), bs*2,False, num_workers=2)
)
# ════════════════════════════════════════════════════════════════════════
# 1 · Function that trains *one* configuration and returns mean AUUC
# ════════════════════════════════════════════════════════════════════════
def eval_cfg(lr, λ_D, λ_F, max_ep=20):
gkf = GroupKFold(n_splits=3)
folds = gkf.split(np.zeros(len(y)), y, groups=domains)
auucs = []
for k,(tr,vl) in enumerate(folds,1):
mdl = DANNSystem(d_in=X.shape[1], λ_D=λ_D, λ_F=λ_F, lr=lr)
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep, accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, log_every_n_steps=5, enable_checkpointing=False,
gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc",mode="max",patience=5)],
)
trainer.fit(mdl, trL, vlL)
best = trainer.callback_metrics["val_auuc"].item()
print(f" fold{k}: best_AUUC={best:.3f}")
auucs.append(best)
del mdl; gc.collect(); torch.cuda.empty_cache()
mean,std = np.mean(auucs), np.std(auucs)
print(f"⇒ mean AUUC={mean:.3f} ±{std:.3f}\n")
return -mean, auucs # Optuna minimizes
# ════════════════════════════════════════════════════════════════════════
# 2 · Quick baseline run
# ════════════════════════════════════════════════════════════════════════
print("Baseline (lr=1e-4, λ_D=1, λ_F=10)")
_, baseline_folds = eval_cfg(lr=1e-4, λ_D=1.0, λ_F=10.0)
# ════════════════════════════════════════════════════════════════════════
# 3 · Optuna sweep
# ════════════════════════════════════════════════════════════════════════
def objective(trial):
lr = trial.suggest_float("lr", 1e-4, 5e-4, log=True)
λ_D = trial.suggest_float("λ_D", .5, 2.0)
λ_F = trial.suggest_float("λ_F", 1.0, 15.0)
return eval_cfg(lr, λ_D, λ_F, max_ep=15)[0]
N_TRIALS = 15
study = optuna.create_study(direction="minimize")
study.optimize(objective, n_trials=N_TRIALS)
best = study.best_params
print(f"★ Best params: {best} → mean_AUUC {(-study.best_value):.3f}")
# ════════════════════════════════════════════════════════════════════════
# 4 · Re-train best config & save figures
# ════════════════════════════════════════════════════════════════════════
gkf = GroupKFold(n_splits=3)
auucs, gap_g, gap_e = [], [], []
for fold,(tr,vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains),1):
mdl = DANNSystem(d_in=X.shape[1], **best)
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(max_epochs=20, devices=1, accelerator="gpu"
if torch.cuda.is_available() else "cpu",
enable_checkpointing=False, gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc",mode="max",patience=5)],
logger=False)
trainer.fit(mdl, trL, vlL)
auuc = trainer.callback_metrics["val_auuc"].item(); auucs.append(auuc)
# ----- Uplift curve -----
u,tv,yv = [],[],[]
for xb,tb,yb,_,_,_ in vlL:
xb = xb.to(mdl.device)
with torch.no_grad():
z = mdl.encoder(xb)
u.append((mdl.head1(z)-mdl.head0(z)).cpu().numpy().ravel())
tv.append(tb.numpy().ravel()); yv.append(yb.numpy().ravel())
u = np.concatenate(u); tv=np.concatenate(tv); yv=np.concatenate(yv)
xs,lift,_ = uplift_curve(yv,tv,u,bins=10)
plt.figure(); plt.plot(xs,lift); plt.xlabel("top-x fraction"); plt.ylabel("net uplift")
plt.title(f"Fold {fold} AUUC={auuc:.3f}"); plt.savefig(f"/kaggle/working/uplift_fold{fold}.png",dpi=140)
# ----- Fairness gaps -----
gap_g.append(fairness_gap(u, (gender[vl]==0)&(tv==1), (gender[vl]==1)&(tv==1)))
gap_e.append(fairness_gap(u, (edu[vl]==0)&(tv==1), (edu[vl]==1)&(tv==1)))
# === Fairness bar-chart ===
labels = np.arange(1,4)
plt.figure(figsize=(6,4))
plt.bar(labels-.17, gap_g, .33, label="gender")
plt.bar(labels+.17, gap_e, .33, label="education"); plt.xticks(labels, [f"Fold {i}" for i in labels])
plt.ylabel("|Δ uplift|"); plt.title("Fairness gaps"); plt.legend()
plt.savefig("/kaggle/working/fairness_gaps.png", dpi=140)
plt.savefig("/kaggle/working/fairness_gaps.pdf", dpi=140)
plt.show()
print("\nFinal per-fold AUUC:", np.round(auucs,3).tolist(),
" mean =", np.mean(auucs).round(3))
print("Figures saved in /kaggle/working/ .")
# %%
import matplotlib.pyplot as plt
import numpy as np
labels = np.array([1, 2, 3])
cohort_names = ["Psychiatric", "Medical", "Quarantine"]
# Values (from above cell's output) follow the corrected fold assignment:
# fold 1 → Psychiatric, fold 2 → Medical, fold 3 → Quarantine
gap_g = [0.11, 0.03, 0.10]
gap_e = [0.13, 0.00, 0.00]
plt.figure(figsize=(8, 6))
plt.bar(labels - 0.17, gap_g, width=0.33, label="Gender", color="#4e79a7")
plt.bar(labels + 0.17, gap_e, width=0.33, label="Education", color="#f28e2b")
plt.ylim(0.000, 0.200)
plt.xticks(labels, cohort_names)
plt.xlabel("Held-out Domain", fontfamily='serif', fontsize=17)
plt.ylabel("Fairness Gap (|Δ uplift|)", fontfamily='serif', fontsize=17)
plt.legend(title="Subgroup")
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
for i in range(len(labels)):
plt.text(labels[i] - 0.17, gap_g[i] + 0.005, f"{gap_g[i]:.2f}", ha='center', va='bottom', fontsize=15)
plt.text(labels[i] + 0.17, gap_e[i] + 0.005, f"{gap_e[i]:.2f}", ha='center', va='bottom', fontsize=15)
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.tight_layout()
plt.savefig("/kaggle/working/fairness_gaps.png", dpi=600)
plt.savefig("/kaggle/working/fairness_gaps.pdf", dpi=600)
plt.show()
# %%
# choose seeds you want to average over
SEEDS = [0, 1, 2, 3, 4] # for example 5 seeds
N_FOLDS = 3 # your LOSO folds
# containers to collect per-seed per-fold outputs
auucs_dann_list, gap_g_dann_list, gap_e_dann_list = [], [], []
auucs_noda_list, gap_g_noda_list, gap_e_noda_list = [], [], []
for s in SEEDS:
# set reproducible seed for this trial
random.seed(s); np.random.seed(s); torch.manual_seed(s)
torch.cuda.manual_seed_all(s)
torch.backends.cudnn.deterministic = True
# run LOSO for DANN and NoDA (returns arrays shape (n_folds,))
auucs_dann_s, gap_g_dann_s, gap_e_dann_s = loso_with_gaps(f"DANN_seed{s}", use_dann=True, max_ep=20)
auucs_noda_s, gap_g_noda_s, gap_e_noda_s = loso_with_gaps(f"NoDA_seed{s}", use_dann=False, max_ep=20)
# append (keep as 1-D arrays length n_folds)
auucs_dann_list.append(np.asarray(auucs_dann_s).ravel())
gap_g_dann_list.append(np.asarray(gap_g_dann_s).ravel())
gap_e_dann_list.append(np.asarray(gap_e_dann_s).ravel())
auucs_noda_list.append(np.asarray(auucs_noda_s).ravel())
gap_g_noda_list.append(np.asarray(gap_g_noda_s).ravel())
gap_e_noda_list.append(np.asarray(gap_e_noda_s).ravel())
# Convert lists-of-arrays into single flattened arrays: length = n_seeds * n_folds
auucs_dann_all = np.concatenate(auucs_dann_list) # shape (n_seeds * n_folds,)
auucs_noda_all = np.concatenate(auucs_noda_list)
gap_g_dann_all = np.concatenate(gap_g_dann_list)
gap_g_noda_all = np.concatenate(gap_g_noda_list)
gap_e_dann_all = np.concatenate(gap_e_dann_list)
gap_e_noda_all = np.concatenate(gap_e_noda_list)
# Optional: per-seed means (for reporting)
auuc_dann_per_seed_mean = np.array([arr.mean() for arr in auucs_dann_list])
auuc_noda_per_seed_mean = np.array([arr.mean() for arr in auucs_noda_list])
print("Aggregated shapes:", auucs_dann_all.shape, gap_g_dann_all.shape, gap_e_dann_all.shape)
print("Per-seed AUUC means (DANN):", auuc_dann_per_seed_mean)
print("Per-seed AUUC means (NoDA):", auuc_noda_per_seed_mean)
# %%
# --- Utility: cliff's delta
def cliffs_delta(x, y):
x = np.asarray(x).ravel()
y = np.asarray(y).ravel()
n_x, n_y = len(x), len(y)
greater = 0
lesser = 0
for xi in x:
greater += np.sum(xi > y)
lesser += np.sum(xi < y)
return (greater - lesser) / (n_x * n_y)
# --- Utility: bootstrap CI for paired median difference
def bootstrap_paired_ci(a, b, stat_func=np.median, n_boot=10000, alpha=0.05, seed=0):
rng = np.random.default_rng(seed)
arr = np.array(a) - np.array(b)
boot_stats = []
n = len(arr)
for _ in range(n_boot):
idx = rng.integers(0, n, n)
boot_stats.append(stat_func(arr[idx]))
lo = np.percentile(boot_stats, 100 * (alpha/2))
hi = np.percentile(boot_stats, 100 * (1 - alpha/2))
return lo, hi
auucs_dann = np.asarray(auucs_dann_all)
auucs_noda = np.asarray(auucs_noda_all)
gap_g_dann = np.asarray(gap_g_dann_all)
gap_g_noda = np.asarray(gap_g_noda_all)
gap_e_dann = np.asarray(gap_e_dann_all)
gap_e_noda = np.asarray(gap_e_noda_all)
n_folds = len(auucs_dann)
fold_labels = [f"Fold {i+1}" for i in range(n_folds)]
# -------------------------
# Summary stats
# -------------------------
def summarize_pair(a, b, name):
mean_a, std_a = a.mean(), a.std(ddof=1)
mean_b, std_b = b.mean(), b.std(ddof=1)
dif = a - b
mean_diff = dif.mean()
std_diff = dif.std(ddof=1)
# Wilcoxon signed-rank test (paired nonparametric)
try:
stat, pval = wilcoxon(a, b)
except Exception:
stat, pval = np.nan, np.nan
# Cliff's delta
cd = cliffs_delta(a, b)
# Bootstrap CI for paired median diff
lo_ci, hi_ci = bootstrap_paired_ci(a, b, stat_func=np.median, n_boot=5000, seed=42)
print(f"\n{name}:")
print(f" mean A = {mean_a:.4f} ± {std_a:.4f}")
print(f" mean B = {mean_b:.4f} ± {std_b:.4f}")
print(f" mean paired diff (A-B) = {mean_diff:.4f} ± {std_diff:.4f}")
print(f" Wilcoxon stat={stat:.4f}, p={pval:.4f}")
print(f" Cliff's delta = {cd:.4f}")
print(f" Paired median bootstrap 95% CI = [{lo_ci:.4f}, {hi_ci:.4f}]")
return dict(mean_a=mean_a, std_a=std_a, mean_b=mean_b, std_b=std_b,
mean_diff=mean_diff, std_diff=std_diff, pval=pval, cd=cd,
median_ci=(lo_ci, hi_ci))
res_auuc = summarize_pair(auucs_dann, auucs_noda, "AUUC (DANN vs NoDA)")
res_gap_g = summarize_pair(gap_g_dann, gap_g_noda, "Gender gap |Δ uplift| (DANN vs NoDA)")
res_gap_e = summarize_pair(gap_e_dann, gap_e_noda, "Education gap |Δ uplift| (DANN vs NoDA)")
# -------------------------
# Plots
# -------------------------
import matplotlib.pyplot as plt
plt.style.use("seaborn-whitegrid")
# 1) Mean AUUC bar with errorbars
fig, ax = plt.subplots(figsize=(4.5,3.5))
means = [auucs_dann.mean(), auucs_noda.mean()]
stds = [auucs_dann.std(ddof=1), auucs_noda.std(ddof=1)]
bars = ax.bar(["DANN", "NoDA"], means, yerr=stds, capsize=6, color=["#4e79a7","#e15759"])
ax.set_ylabel("Mean LOSO-AUUC")
ax.set_title("Mean LOSO-AUUC: DANN vs NoDA")
for i, m in enumerate(means):
ax.text(i, m + 0.02*max(means), f"{m:.3f}", ha="center")
plt.tight_layout()
plt.savefig("/kaggle/working/mean_loso_auuc_DANN_vs_NoDA.png", dpi=300)
plt.show()
plt.close(fig)
# 2) Grouped fairness bars per fold (gender)
width = 0.35
x = np.arange(n_folds)
fig, ax = plt.subplots(figsize=(6,3.2))
ax.bar(x - width/2, gap_g_dann, width, label="DANN", color="#4e79a7")
ax.bar(x + width/2, gap_g_noda, width, label="NoDA", color="#59a14f")
ax.set_xticks(x); ax.set_xticklabels(fold_labels)
ax.set_ylabel("|Δ uplift| (gender)"); ax.set_title("Gender fairness: DANN vs NoDA")
ax.legend(); plt.tight_layout()
plt.savefig("/kaggle/working/gender_gap_DANN_vs_NoDA_per_fold.png", dpi=300)
plt.show()
plt.close(fig)
# 3) Grouped fairness bars per fold (education)
fig, ax = plt.subplots(figsize=(6,3.2))
ax.bar(x - width/2, gap_e_dann, width, label="DANN", color="#4e79a7")
ax.bar(x + width/2, gap_e_noda, width, label="NoDA", color="#59a14f")
ax.set_xticks(x); ax.set_xticklabels(fold_labels)
ax.set_ylabel("|Δ uplift| (education)"); ax.set_title("Education fairness: DANN vs NoDA")
ax.legend(); plt.tight_layout()
plt.savefig("/kaggle/working/edu_gap_DANN_vs_NoDA_per_fold.png", dpi=300)
plt.show()
plt.close(fig)
# 4) Combined summary panel (AUUC, mean gender gap, mean edu gap)
fig, axes = plt.subplots(1,3, figsize=(12,3.5), gridspec_kw={"width_ratios":[1,1,1]})
# AUUC
axes[0].bar(["DANN","NoDA"], means, yerr=stds, color=["#4e79a7","#e15759"], capsize=6)
axes[0].set_title("Mean LOSO-AUUC"); axes[0].set_ylabel("AUUC")
# Gender mean gaps
axes[1].bar(["DANN","NoDA"], [gap_g_dann.mean(), gap_g_noda.mean()], color=["#4e79a7","#59a14f"])
axes[1].set_title("Mean Gender gap"); axes[1].set_ylabel("|Δ uplift|")
# Education mean gaps
axes[2].bar(["DANN","NoDA"], [gap_e_dann.mean(), gap_e_noda.mean()], color=["#4e79a7","#59a14f"])
axes[2].set_title("Mean Education gap"); axes[2].set_ylabel("|Δ uplift|")
plt.suptitle("DANN vs NoDA: AUUC and Fairness diagnostics")
plt.tight_layout(rect=[0,0,1,0.95])
plt.savefig("/kaggle/working/DANN_vs_NoDA_summary.png", dpi=300)
plt.show()
plt.close(fig)
# 5) Paired difference histogram for AUUC
fig, ax = plt.subplots(figsize=(5,3.2))
diffs = auucs_dann - auucs_noda
ax.hist(diffs, bins=8, color="#4e79a7", edgecolor="k")
ax.axvline(0, color="k", linestyle="--")
ax.set_xlabel("AUUC(DANN) - AUUC(NoDA)"); ax.set_title("Distribution of paired AUUC differences")
plt.tight_layout()
plt.savefig("/kaggle/working/paired_auuc_diff_hist.png", dpi=300)
plt.show()
plt.close(fig)
print("\nPlots saved under /kaggle/working/.")
# %%
# ======================================================================
# 0 · Reproducible seed
# ======================================================================
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
torch.cuda.manual_seed_all(SEED); torch.backends.cudnn.deterministic = True
# ======================================================================
# 1 · Helper: build one model with/without domain loss
# ======================================================================
def build_model(use_dann: bool,
lr = 1e-4,
λ_D = 1.0,
λ_F = 10.0):
"""Return a DANN_Uplift with λ_D = 0 (NoDA) or λ_D > 0 (DANN)."""
λ_D_eff = λ_D if use_dann else 0.0
return DANN_Uplift(d_in=X.shape[1],
λ_D=λ_D_eff,
λ_F=λ_F,
lr=lr)
# ======================================================================
# 2 · LOSO evaluation (unchanged loaders / trainer settings)
# ======================================================================
def loso_scores(tag, use_dann, max_ep=20):
gkf = GroupKFold(n_splits=3)
scores = []
for k,(tr,vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains),1):
mdl = build_model(use_dann=use_dann)
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1,
enable_checkpointing=False,
callbacks=[pl.callbacks.EarlyStopping("val_auuc",
mode="max",
patience=5)],
logger=False,
log_every_n_steps=5,
)
trainer.fit(mdl, trL, vlL)
auuc = trainer.callback_metrics["val_auuc"].item()
print(f"{tag} fold{k}: AUUC = {auuc:+.3f}")
scores.append(auuc)
del mdl; gc.collect(); torch.cuda.empty_cache()
print(f"{tag} mean AUUC = {np.mean(scores):+.3f}\n")
return scores
# ======================================================================
# 3 · Run the comparison (same seed, same hyper-params)
# ======================================================================
scores_dann = loso_scores("DANN-128-64-d0.30", use_dann=True)
scores_noda = loso_scores("NoDA-128-64-d0.30", use_dann=False)
# Simple bar-plot
import matplotlib.pyplot as plt
plt.bar(["DANN", "NoDA"],
[np.mean(scores_dann), np.mean(scores_noda)],
color=["#4e79a7", "#e15759"])
plt.ylabel("Mean LOSO-AUUC")
plt.title("128-64 encoder, dropout 0.30")
plt.savefig("/kaggle/working/loso_auUc_DANNvsNoDA.png", dpi=500)
plt.savefig("/kaggle/working/loso_auUc_DANNvsNoDA.pdf", dpi=500)
plt.show()
# %%
import matplotlib.pyplot as plt
plt.figure(figsize=(5, 5))
plt.bar(["DANN", "NoDA"],
[np.mean(scores_dann), np.mean(scores_noda)],
color=["#4e79a7", "#e15759"])
plt.xlabel("Model Type",fontfamily='serif',fontsize=16)
plt.ylabel("Mean LOSO-AUUC",fontfamily='serif',fontsize=16)
# plt.title("128-64 encoder, dropout 0.30")
# plt.grid(True, linestyle=':', linewidth=0.75, color='black')
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
plt.tight_layout()
plt.savefig("/kaggle/working/loso_AUUC_DANNvsNoDA.png", dpi=600)
plt.savefig("/kaggle/working/loso_AUUC_DANNvsNoDA.pdf", dpi=600)
plt.show()
# %% [markdown]
# # λD sweep: Confirms λD is a leverage knob for cross-domain benefit only when the causal sign is correct
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
# ------------------------------------------------------------------
# λ_D grid sweep (per-fold AUUC)
# ------------------------------------------------------------------
lambdas = np.arange(0.1, 0.9, 0.1) # 0.1 … 0.8
grid_res = [] # rows: (λ_D, fold, AUUC)
for lam in lambdas:
mean_neg, folds = eval_cfg(lr=2e-4, λ_D=lam, λ_F=5.0, max_ep=15) # keep lr & λ_F fixed
for k, score in enumerate(folds, 1):
grid_res.append({"λ_D": lam, "fold": k, "AUUC": score})
torch.cuda.empty_cache(); gc.collect()
df_grid = pd.DataFrame(grid_res)
# ---------- Plot ----------
plt.figure(figsize=(6,4))
for k in [1,2,3]:
plt.plot(df_grid.query("fold==@k")["λ_D"],
df_grid.query("fold==@k")["AUUC"],
marker="o", label=f"Fold {k}")
plt.axhline(0, color="grey", ls="--", lw=0.7)
plt.xlabel("$\\lambda_D$")
plt.ylabel("LOSO-AUUC")
plt.title("λ_D sweep (encoder 128-64, d=0.30)")
plt.legend();
plt.savefig("/kaggle/working/λ_D sweep.png", dpi=500)
plt.savefig("/kaggle/working/λ_D sweep.pdf", dpi=500)
plt.tight_layout(); plt.show()
# %%
# Values taken from the output of above cell's output
lambda_d = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]
data = {
"λ_D": lambda_d * 3,
"fold": [1]*8 + [2]*8 + [3]*8,
"AUUC": [
# fold 1 → Psychiatric (corrected)
-3.688, -2.211, -3.900, -2.318, -2.460, -2.497, -2.874, -2.840,
# fold 2 → Medical (unchanged)
1.964, 0.896, 1.867, 1.807, 3.693, -0.390, 1.249, 2.655,
# fold 3 → Quarantine (corrected)
0.371, 1.125, -0.055, 2.729, 2.172, 0.376, 0.201, 2.451,
]
}
df_grid = pd.DataFrame(data)
plt.figure(figsize=(6, 4))
fold_names = {1: "Psychiatric", 2: "Medical", 3: "Quarantine"}
for k in [1, 2, 3]:
subset = df_grid.query("fold == @k")
plt.plot(subset["λ_D"], subset["AUUC"], marker="o", label=fold_names[k])
plt.xlabel("Adversarial weight λ_D", fontfamily='serif', fontsize=16)
plt.ylabel("LODO-AUUC", fontfamily='serif', fontsize=16)
plt.legend(title="Held-out domain")
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
plt.tight_layout()
plt.savefig("/kaggle/working/auuc_lambdaD_sweep.png", dpi=600)
plt.savefig("/kaggle/working/auuc_lambdaD_sweep.pdf", dpi=600)
plt.show()
# %% [markdown]
# # Uplift curve on the Psychiatric test fold. The curve lies below the random baseline (AUUC ≈−2.66), indicating an inverted treatment-effect ranking.
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
FOLD_BAD = 1
# Get the idx splits exactly as eval_cfg does
gkf = GroupKFold(n_splits=3)
splits = list(gkf.split(np.zeros(len(y)), y, groups=domains))
tr_idx, vl_idx = splits[FOLD_BAD-1]
# Re-train one model *only for diagnostics*
best_lam = 0.4 # put the λ_D
mdl = DANNSystem(d_in=X.shape[1], λ_D=best_lam, λ_F=5.0, lr=2e-4)
trL, vlL = make_loaders(tr_idx, vl_idx, bs=128)
pl.Trainer(max_epochs=20, devices=1, enable_checkpointing=False,
logger=False).fit(mdl, trL, vlL)
# ---------- uplifts & curve ----------
u, tv, yv = [], [], []
for xb,tb,yb,_,_,_ in vlL:
xb = xb.to(mdl.device)
with torch.no_grad():
z = mdl.encoder(xb)
u.append((mdl.head1(z)-mdl.head0(z)).cpu().numpy().ravel())
tv.append(tb.numpy().ravel()); yv.append(yb.numpy().ravel())
u, tv, yv = map(np.concatenate, [u, tv, yv])
xs, lift, area = uplift_curve(yv, tv, u, bins=10)
plt.figure(figsize=(6,4))
plt.plot(xs, lift, marker="o")
plt.axhline(0, color="grey", ls="--", lw=0.7)
plt.xlabel("top-x fraction"); plt.ylabel("net uplift")
plt.subplots_adjust(top=0.9)
plt.savefig("/kaggle/working/netupliftcurve_Psychiatric.png", dpi=500)
plt.savefig("/kaggle/working/netupliftcurve_Psychiatric.pdf", dpi=500)
plt.show()
# ---------- sanity checks ----------
print("T-rate in fold:", tv.mean().round(3))
print("Mean uplift sign:", np.sign(u).mean())
# %%
plt.figure(figsize=(6,3))
plt.plot(xs, lift, marker="o")
plt.axhline(0, color="grey", ls="--", lw=0.7)
plt.xlabel("Top-x fraction",fontfamily='serif', fontsize=16); plt.ylabel("Net uplift",fontfamily='serif', fontsize=16)
plt.ylim(-0.35, -0.250)
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
plt.tight_layout();
plt.savefig("/kaggle/working/netupliftcurve_Psychiatric.pdf", dpi=600)
plt.show()
# %% [markdown]
# # Scheduled λD (GRL weight)- Linear, Sigmoid vs. fixed
# %%
# ───────────────────────────────────────────────────────────────
# 1 · Function that trains *one* configuration and returns mean AUUC
# + added model_cls hook so we can swap in scheduled-λ models
# ───────────────────────────────────────────────────────────────
def eval_cfg(lr, λ_D, λ_F, max_ep=20, # ← old args
model_cls=DANNSystem): # ← NEW arg, default stays DANN
gkf = GroupKFold(n_splits=3)
folds = gkf.split(np.zeros(len(y)), y, groups=domains)
auucs = []
for k, (tr, vl) in enumerate(folds, 1):
mdl = model_cls(d_in=X.shape[1], λ_D=λ_D, λ_F=λ_F, lr=lr)
trL, vL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, log_every_n_steps=5,
enable_checkpointing=False, gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc", mode="max", patience=5)],
)
trainer.fit(mdl, trL, vL)
best = trainer.callback_metrics["val_auuc"].item()
print(f" fold{k}: best_AUUC={best:+.3f}")
auucs.append(best)
del mdl; gc.collect(); torch.cuda.empty_cache()
mean, std = np.mean(auucs), np.std(auucs)
print(f"⇒ mean AUUC={mean:+.3f} ±{std:.3f}\n")
return -mean, auucs # Optuna still minimises
# %%
from functools import partial
# ───────────────────────────────────────────────────────────────
# 4 · Scheduled λ_D (linear & sigmoid) ── replaces fixed scalar
# ───────────────────────────────────────────────────────────────
def lambda_linear(p, max_lambda=0.6):
"""Linear ramp from 0 → maxLambda over first half of training."""
return min(1.0, 2*p) * max_lambda # p = epoch / max_epochs
def lambda_sigmoid(p, max_lambda=0.6, k=10):
"""Sigmoid ramp (original DANN paper style)."""
return max_lambda * 1/(1+np.exp(-k*(p-0.5)))
class DANN_Uplift_Scheduled(DANN_Uplift):
def __init__(self, d_in, schedule="linear", **kwargs):
super().__init__(d_in, **kwargs) # inherits λ_D as *max*
self.schedule = schedule
# override training_step to update GRL weight each epoch
def on_train_epoch_start(self):
prog = self.current_epoch / self.trainer.max_epochs
if self.schedule == "linear":
self.grl.l = lambda_linear(prog, self.hparams.λ_D)
else:
self.grl.l = lambda_sigmoid(prog, self.hparams.λ_D)
# helper to run one schedule
def run_schedule(schedule):
print(f"\n=== {schedule} schedule (λ_D max={best_lambda_d:.2f}) ===")
mean_neg,_ = eval_cfg(lr=best_lr, λ_D=best_lambda_d, λ_F=best_lambda_f,
max_ep=20, model_cls=partial(DANN_Uplift_Scheduled,
schedule=schedule))
print(f"Mean AUUC = {-mean_neg:.3f}")
return -mean_neg
# %%
# Use the best lr / λ_F from last Optuna run
best_lr, best_lambda_f, best_lambda_d = 2e-4,5.0,0.8 # Going with LAMBDA_D=0.8
fixed = -eval_cfg(best_lr, best_lambda_d, best_lambda_f, max_ep=20)[0]
lin = run_schedule("linear")
sig = run_schedule("sigmoid")
print(f"\nFixed={fixed:.3f} Linear={lin:.3f} Sigmoid={sig:.3f}")
# %%
# Use the best lr / λ_F from last Optuna run
best_lr, best_lambda_f, best_lambda_d = 2e-4,5.0,0.7 # Going with LAMBDA_D=0.7
fixed = -eval_cfg(best_lr, best_lambda_d, best_lambda_f, max_ep=20)[0]
lin = run_schedule("linear")
sig = run_schedule("sigmoid")
print(f"\nFixed={fixed:.3f} Linear={lin:.3f} Sigmoid={sig:.3f}")
# %%
# Use the best lr / λ_F from last Optuna run
best_lr, best_lambda_f, best_lambda_d = 2e-4,5.0,0.8 # Going with LAMBDA_D=0.8
fixed = -eval_cfg(best_lr, best_lambda_d, best_lambda_f, max_ep=20)[0]
lin = run_schedule("linear")
sig = run_schedule("sigmoid")
print(f"\nFixed={fixed:.3f} Linear={lin:.3f} Sigmoid={sig:.3f}")
# %% [markdown]
# # Single-Domain sanity check
# %%
# ───────────────────────────────────────────────────────────────
# 5 · Train-on-X, test-on-Y (no adaptation) for every pair
# ───────────────────────────────────────────────────────────────
pairs = [(0,1), (0,2), (1,0), (1,2), (2,0), (2,1)] # med↔quar↔psych
pair_scores = {}
for src, tgt in pairs:
tr_idx = np.where(domains == src)[0]
te_idx = np.where(domains == tgt)[0]
trL, vlL = make_loaders(tr_idx, te_idx, bs=128)
base = DANN_Uplift(d_in=X.shape[1], λ_D=0.0, λ_F=0.0, lr=best_lr)
trainer = pl.Trainer(max_epochs=15, enable_checkpointing=False,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, logger=False)
trainer.fit(base, trL, vlL)
auuc = trainer.callback_metrics['val_auuc'].item()
pair_scores[(src, tgt)] = auuc
print(f"train {src} → test {tgt} AUUC={auuc:+.3f}")
# Compare against same split using best-tuned DANN
print("\nRepeating with best-tuned DANN:")
for src, tgt in pairs:
tr_idx = np.where(domains == src)[0]
te_idx = np.where(domains == tgt)[0]
trL, vlL = make_loaders(tr_idx, te_idx, bs=128)
dann = DANN_Uplift(d_in=X.shape[1], λ_D=best_lambda_d,
λ_F=best_lambda_f, lr=best_lr)
trainer = pl.Trainer(max_epochs=15, enable_checkpointing=False,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, logger=False)
trainer.fit(dann, trL, vlL)
auuc = trainer.callback_metrics['val_auuc'].item()
print(f"DANN train {src} → test {tgt} AUUC={auuc:+.3f}")
# %% [markdown]
# # Semi-Synthetic benchmark (CATE RMSE)
# %%
# ───────────────────────────────────────────────────────────────
# 6 · Semi-synthetic: add synthetic uplift +0.30 for age<25, –0.10 else
# ───────────────────────────────────────────────────────────────
def make_semi_synth(df):
df2 = df.copy()
# true CATE δ(x): young → +0.3, older → –0.1
delta = np.where(df2['age'] < 25, 0.3, -0.1)
# generate synthetic outcome: y = y_orig ⊕ Bernoulli(δ * T)
noise = np.random.binomial(1, np.abs(delta)*df2['T'])
df2['Y_synth'] = (df2['Y'] ^ noise).astype(int)
df2['delta_true'] = delta
return df2
synth_df = make_semi_synth(df_all)
# … repeat pre-processing to get X_synth, y_synth, etc.
# Train DANN on synthetic labels
mdl = DANN_Uplift(d_in=X.shape[1], λ_D=best_lambda_d,
λ_F=0.0, lr=best_lr)
gkf = GroupKFold(3)
rmse = []
for tr, vl in gkf.split(np.zeros(len(synth_df)), y, groups=domains):
trL, vlL = make_loaders(tr, vl, bs=128)
trainer = pl.Trainer(max_epochs=15, enable_checkpointing=False,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, logger=False)
trainer.fit(mdl, trL, vlL)
preds = []
for xb,tb in DataLoader(torch.utils.data.TensorDataset(
torch.tensor(X[vl],dtype=torch.float32),
torch.tensor(T[vl],dtype=torch.float32)), batch_size=256):
xb,tb = xb.to(mdl.device), tb.to(mdl.device)
with torch.no_grad():
y0,y1,_ = mdl(xb,tb)
preds.append((y1-y0).cpu().numpy())
uplift_hat = np.concatenate(preds).ravel()
rmse.append(np.sqrt(((uplift_hat - synth_df['delta_true'].iloc[vl])**2).mean()))
print("Semi-synthetic CATE-RMSE per fold:", np.round(rmse,3),
" mean =", np.mean(rmse).round(3))
# %%
# ───────────────────────────────────────────────────────────────
# 6 · Semi-synthetic: add synthetic uplift +0.30 for age<25, –0.10 else
# ───────────────────────────────────────────────────────────────
def make_semi_synth(df):
df2 = df.copy()
# true CATE δ(x): young → +0.3, older → –0.1
delta = np.where(df2['age'] < 25, 0.3, -0.1)
# generate synthetic outcome: y = y_orig ⊕ Bernoulli(δ * T)
noise = np.random.binomial(1, np.abs(delta)*df2['T'])
df2['Y_synth'] = (df2['Y'] ^ noise).astype(int)
df2['delta_true'] = delta
return df2
synth_df = make_semi_synth(df_all)
# … repeat pre-processing to get X_synth, y_synth, etc.
# Train DANN on synthetic labels
mdl = DANN_Uplift(d_in=X.shape[1], λ_D=best_lambda_d,
λ_F=0.0, lr=best_lr)
gkf = GroupKFold(3)
rmse = []
for tr, vl in gkf.split(np.zeros(len(synth_df)), y, groups=domains):
trL, vlL = make_loaders(tr, vl, bs=128)
trainer = pl.Trainer(max_epochs=15, enable_checkpointing=False,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, logger=False)
trainer.fit(mdl, trL, vlL)
preds = []
for xb,tb in DataLoader(torch.utils.data.TensorDataset(
torch.tensor(X[vl],dtype=torch.float32),
torch.tensor(T[vl],dtype=torch.float32)), batch_size=256):
xb,tb = xb.to(mdl.device), tb.to(mdl.device)
with torch.no_grad():
y0,y1,_ = mdl(xb,tb)
preds.append((y1-y0).cpu().numpy())
uplift_hat = np.concatenate(preds).ravel()
rmse.append(np.sqrt(((uplift_hat - synth_df['delta_true'].iloc[vl])**2).mean()))
print("Semi-synthetic CATE-RMSE per fold:", np.round(rmse,3),
" mean =", np.mean(rmse).round(3))
# %% [markdown]
# # Comparison
# %%
# ---------- CORAL (Deep CORAL) ----------
def coral_loss(z: torch.Tensor, dom: torch.Tensor) -> torch.Tensor:
# z: (B, d), dom: (B,)
uniq = dom.unique()
if len(uniq) < 2:
return torch.zeros(1, device=z.device)
losses = []
for i in range(len(uniq)-1):
zi = z[dom == uniq[i]]
for j in range(i+1, len(uniq)):
zj = z[dom == uniq[j]]
if zi.shape[0] < 2 or zj.shape[0] < 2:
continue
# mean & covariance
mi, mj = zi.mean(0, keepdim=True), zj.mean(0, keepdim=True)
ci = torch.cov(zi.T) if zi.shape[0] > 1 else torch.zeros((z.shape[1], z.shape[1]), device=z.device)
cj = torch.cov(zj.T) if zj.shape[0] > 1 else torch.zeros((z.shape[1], z.shape[1]), device=z.device)
losses.append((mi - mj).pow(2).mean() + (ci - cj).pow(2).mean())
return sum(losses)/len(losses) if losses else torch.zeros(1, device=z.device)
# ---------- MMD (RBF kernel with median heuristic) ----------
def _rbf_mmd(x: torch.Tensor, y: torch.Tensor) -> torch.Tensor:
# pairwise squared distances
xx = torch.cdist(x, x, p=2).pow(2)
yy = torch.cdist(y, y, p=2).pow(2)
xy = torch.cdist(x, y, p=2).pow(2)
# median heuristic for gamma
with torch.no_grad():
med = torch.median(xy)
gamma = 1.0 / (med + 1e-6)
Kxx = torch.exp(-gamma * xx)
Kyy = torch.exp(-gamma * yy)
Kxy = torch.exp(-gamma * xy)
return Kxx.mean() + Kyy.mean() - 2.0 * Kxy.mean()
def mmd_loss(z: torch.Tensor, dom: torch.Tensor) -> torch.Tensor:
uniq = dom.unique()
if len(uniq) < 2:
return torch.zeros(1, device=z.device)
losses = []
for i in range(len(uniq)-1):
zi = z[dom == uniq[i]]
for j in range(i+1, len(uniq)):
zj = z[dom == uniq[j]]
if zi.shape[0] < 2 or zj.shape[0] < 2:
continue
losses.append(_rbf_mmd(zi, zj))
return sum(losses)/len(losses) if losses else torch.zeros(1, device=z.device)
# %%
class CDAN_Uplift(DANN_Uplift):
"""Conditional DANN: discriminator sees features + predicted heads."""
def __init__(self, d_in, **kwargs):
super().__init__(d_in, **kwargs)
# infer encoder output size (64 in your 128-64 encoder; 16 in 32-16)
z_dim = self.head0.in_features
# rebuild discriminator to accept (z_dim + 2)
self.disc = nn.Sequential(
nn.Linear(z_dim + 2, 64), nn.ReLU(), nn.Linear(64, 3)
)
def forward(self, x, t):
z = self.encoder(x)
y0 = torch.sigmoid(self.head0(z))
y1 = torch.sigmoid(self.head1(z))
joint = torch.cat([z, y0, y1], dim=1) # (B, z_dim+2)
d_hat = self.disc(self.grl(joint))
return y0, y1, d_hat
class CORAL_Uplift(DANN_Uplift):
"""Same as your model but domain penalty = CORAL (no discriminator used)."""
def training_step(self, batch, *_):
x, t, y, dom, g, e = batch
z = self.encoder(x)
y0 = torch.sigmoid(self.head0(z))
y1 = torch.sigmoid(self.head1(z))
# primary loss
eps = 1e-7
y_pred = torch.clip(torch.where(t == 1, y1, y0), eps, 1 - eps)
L_up = F.binary_cross_entropy(y_pred, y)
# CORAL alignment
L_dom = coral_loss(z, dom)
# fairness
uplift = (y1 - y0).flatten()
def safe_gap(ma, mb):
return ((uplift[ma].mean() - uplift[mb].mean())**2
if ma.any() and mb.any() else torch.zeros(1, device=x.device))
L_fair = safe_gap(g.flatten()==0, g.flatten()==1) + safe_gap(e.flatten()==0, e.flatten()==1)
loss = L_up + self.hparams.λ_D * L_dom + self.hparams.λ_F * L_fair
self.log_dict({"train_loss": loss, "train_uplift": L_up, "train_dom": L_dom, "train_fair": L_fair},
prog_bar=True, on_epoch=True)
return loss
class MMD_Uplift(DANN_Uplift):
"""Same as your model but domain penalty = MMD (no discriminator used)."""
def training_step(self, batch, *_):
x, t, y, dom, g, e = batch
z = self.encoder(x)
y0 = torch.sigmoid(self.head0(z))
y1 = torch.sigmoid(self.head1(z))
eps = 1e-7
y_pred = torch.clip(torch.where(t == 1, y1, y0), eps, 1 - eps)
L_up = F.binary_cross_entropy(y_pred, y)
L_dom = mmd_loss(z, dom)
uplift = (y1 - y0).flatten()
def safe_gap(ma, mb):
return ((uplift[ma].mean() - uplift[mb].mean())**2
if ma.any() and mb.any() else torch.zeros(1, device=x.device))
L_fair = safe_gap(g.flatten()==0, g.flatten()==1) + safe_gap(e.flatten()==0, e.flatten()==1)
loss = L_up + self.hparams.λ_D * L_dom + self.hparams.λ_F * L_fair
self.log_dict({"train_loss": loss, "train_uplift": L_up, "train_dom": L_dom, "train_fair": L_fair},
prog_bar=True, on_epoch=True)
return loss
# %%
def eval_cfg_model(model_cls, lr, λ_D, λ_F, max_ep=10):
gkf = GroupKFold(n_splits=3)
auucs = []
for k,(tr,vl) in enumerate(gkf.split(np.zeros(len(y)), y, groups=domains),1):
mdl = model_cls(d_in=X.shape[1], λ_D=λ_D, λ_F=λ_F, lr=lr)
trL, vlL = make_loaders(tr, vl)
trainer = pl.Trainer(
max_epochs=max_ep,
accelerator="gpu" if torch.cuda.is_available() else "cpu",
devices=1, log_every_n_steps=5, enable_checkpointing=False,
gradient_clip_val=1.0,
callbacks=[pl.callbacks.EarlyStopping("val_auuc", mode="max", patience=5)],
)
trainer.fit(mdl, trL, vlL)
best = trainer.callback_metrics["val_auuc"].item()
print(f"[{model_cls.__name__}] fold{k}: AUUC={best:.3f}")
auucs.append(best)
del mdl; gc.collect(); torch.cuda.empty_cache()
mean,std = float(np.mean(auucs)), float(np.std(auucs))
print(f"⇒ {model_cls.__name__} mean AUUC={mean:.3f} ±{std:.3f}\n")
return mean, auucs
# %%
RESULTS_FILE = "da_results.json"
# Load existing results if available
if os.path.exists(RESULTS_FILE):
with open(RESULTS_FILE, "r") as f:
results = json.load(f)
else:
results = {}
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
LR = 2e-4
LAMF = 3.0
grid = [0.1, 0.2, 0.4, 0.6]
# Choose one model to run per session
target_cls = DANNSystem
if target_cls.__name__ not in results:
best_mean, best_cfg = -1e9, None
for lamD in grid:
mean, _ = eval_cfg_model(target_cls, lr=LR, λ_D=lamD, λ_F=LAMF, max_ep=15)
if mean > best_mean:
best_mean, best_cfg = mean, {"λ_D": lamD}
results[target_cls.__name__] = {"mean": best_mean, "best": best_cfg}
# Save updated results
with open(RESULTS_FILE, "w") as f:
json.dump(results, f, indent=2)
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
LR = 2e-4
LAMF = 3.0
grid = [0.1, 0.2, 0.4, 0.6]
# Choose one model to run per session
target_cls = CDAN_Uplift
if target_cls.__name__ not in results:
best_mean, best_cfg = -1e9, None
for lamD in grid:
mean, _ = eval_cfg_model(target_cls, lr=LR, λ_D=lamD, λ_F=LAMF, max_ep=15)
if mean > best_mean:
best_mean, best_cfg = mean, {"λ_D": lamD}
results[target_cls.__name__] = {"mean": best_mean, "best": best_cfg}
# Save updated results
with open(RESULTS_FILE, "w") as f:
json.dump(results, f, indent=2)
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
LR = 2e-4
LAMF = 3.0
grid = [0.1, 0.2, 0.4, 0.6]
# Choose one model to run per session
target_cls = CORAL_Uplift
if target_cls.__name__ not in results:
best_mean, best_cfg = -1e9, None
for lamD in grid:
mean, _ = eval_cfg_model(target_cls, lr=LR, λ_D=lamD, λ_F=LAMF, max_ep=15)
if mean > best_mean:
best_mean, best_cfg = mean, {"λ_D": lamD}
results[target_cls.__name__] = {"mean": best_mean, "best": best_cfg}
# Save updated results
with open(RESULTS_FILE, "w") as f:
json.dump(results, f, indent=2)
# %%
SEED = 42
random.seed(SEED); np.random.seed(SEED); torch.manual_seed(SEED)
LR = 2e-4
LAMF = 3.0
grid = [0.1, 0.2, 0.4, 0.6]
# Choose one model to run per session
target_cls = MMD_Uplift
if target_cls.__name__ not in results:
best_mean, best_cfg = -1e9, None
for lamD in grid:
mean, _ = eval_cfg_model(target_cls, lr=LR, λ_D=lamD, λ_F=LAMF, max_ep=15)
if mean > best_mean:
best_mean, best_cfg = mean, {"λ_D": lamD}
results[target_cls.__name__] = {"mean": best_mean, "best": best_cfg}
# Save updated results
with open(RESULTS_FILE, "w") as f:
json.dump(results, f, indent=2)
# %%
if "NoDA" not in results:
nodA_mean, _ = eval_cfg_model(DANNSystem, lr=LR, λ_D=0.0, λ_F=LAMF, max_ep=15)
results["NoDA"] = {"mean": nodA_mean, "best": {"λ_D": 0.0}}
with open(RESULTS_FILE, "w") as f:
json.dump(results, f, indent=2)
# %%
METHODS = {
"NoDA": (DANNSystem, {"λ_D": [0.0]}),
"DANN": (DANNSystem, {"λ_D": [0.2,0.4,0.6]}),
"CDAN": (CDAN_Uplift, {"λ_D": [0.2,0.4,0.6]}),
"CORAL": (CORAL_Uplift, {"λ_D": [0.1,0.5,1.0]}),
"MMD": (MMD_Uplift, {"λ_D": [0.1,0.5,1.0]}),
}
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
GRID = []
for name,(cls,hp) in METHODS.items():
for lr in LRS:
for lamF in LAMFs:
for lamD in hp["λ_D"]:
GRID.append((name,cls,{"lr":lr, "λ_F":lamF, "λ_D":lamD}))
best = {}
for name,cls,cfg in GRID:
mean,_ = eval_cfg_model(cls, max_ep=20, **cfg)
cur = best.get(name, {"mean": -1e9})
if mean > cur["mean"]:
best[name] = {"mean": mean, "cfg": cfg}
print("Best per method:", best)
# %%
# ============ Robust, fair comparison across methods ============
import numpy as np, random, torch, optuna
from optuna.samplers import TPESampler
from collections import defaultdict
from scipy.stats import wilcoxon
import pickle, traceback
SEEDS = list(range(5)) # 5 seeds
LR_GRID = [1e-4, 2e-4]
LAMF_GRID = [0, 1, 3]
LAMD_GRID = [0.2, 0.4, 0.6] # same for all methods
METHODS = {
"NoDA": DANNSystem, # we’ll set λ_D=0 when evaluating
"DANN": DANNSystem,
"CDAN": CDAN_Uplift,
"CORAL": CORAL_Uplift,
"MMD": MMD_Uplift,
}
def set_all_seeds(s):
random.seed(s); np.random.seed(s); torch.manual_seed(s)
torch.cuda.manual_seed_all(s)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
def eval_one(method_cls, lr, lam_f, lam_d, seed, max_ep=10):
set_all_seeds(seed)
return eval_cfg_model(method_cls, lr=lr, λ_D=lam_d, λ_F=lam_f, max_ep=max_ep)
def best_on_grid(method_name, seed):
best_mean, best_cfg = -1e9, None
cls = METHODS[method_name]
for lr in LR_GRID:
for lam_f in LAMF_GRID:
for lam_d in ([0.0] if method_name=="NoDA" else LAMD_GRID):
mean, _ = eval_one(cls, lr, lam_f, lam_d, seed, max_ep=15)
if mean > best_mean:
best_mean = mean
best_cfg = {"lr": lr, "λ_F": lam_f, "λ_D": lam_d}
return best_mean, best_cfg
# Collect per-seed, per-fold AUUC for the *chosen* config per method
records = defaultdict(list) # method -> list of AUUC across (seeds × folds)
def checkpoint():
with open("/kaggle/working/checkpoint.pkl","wb") as f:
pickle.dump({"records":records, "configs":configs}, f)
configs = {} # method -> last best config (for reporting)
for s in SEEDS:
try:
for m in METHODS:
best_mean, cfg = best_on_grid(m, s)
_, folds = eval_cfg_model(METHODS[m], lr=cfg["lr"], λ_D=cfg["λ_D"], λ_F=cfg["λ_F"], max_ep=10)
records[m].extend(folds)
configs[m] = cfg
checkpoint()
except Exception as e:
print("Seed loop crashed:", e)
traceback.print_exc()
checkpoint()
continue
# Summaries
def mean_ci(xs):
xs = np.array(xs)
mu = xs.mean()
se = xs.std(ddof=1) / np.sqrt(len(xs))
ci95 = 1.96 * se
return mu, ci95, xs.std(ddof=1)
for m in METHODS:
mu, ci, sd = mean_ci(records[m])
print(f"{m:5s} mean={mu:+.3f} ±95%CI {ci:.3f} (sd={sd:.3f}) best_cfg≈{configs[m]}")
# Paired tests on matched seed×fold pairs, e.g., DANN vs NoDA
def paired_p(a, b):
return wilcoxon(np.array(a), np.array(b), zero_method="zsplit", correction=True).pvalue
for comp in [("DANN","NoDA"), ("CDAN","DANN"), ("CORAL","DANN"), ("MMD","DANN")]:
p = paired_p(records[comp[0]], records[comp[1]])
print(f"{comp[0]} vs {comp[1]} Wilcoxon p={p:.3f}")
# %%
import os, json, numpy as np, time
SEEDS = list(range(5)) # or 10
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
METHODS = {
"NoDA": (DANNSystem, [0.0]),
"DANN": (DANNSystem, [0.2, 0.4, 0.6]),
"CDAN": (CDAN_Uplift, [0.2, 0.4, 0.6]),
"CORAL": (CORAL_Uplift,[0.1, 0.5, 1.0]),
"MMD": (MMD_Uplift, [0.1, 0.5, 1.0]),
}
RUN_METHOD = "DANN" # <<< change and rerun per method
cls, lamDs = METHODS[RUN_METHOD]
FOLDS_JL = f"/kaggle/working/folds_{RUN_METHOD}.jsonl"
done_seeds = set()
if os.path.exists(FOLDS_JL):
with open(FOLDS_JL) as f:
for line in f:
obj = json.loads(line)
done_seeds.add(obj["seed"])
for seed in SEEDS:
if seed in done_seeds:
print(f"[{RUN_METHOD}] seed {seed} already done; skipping.")
continue
best_mean, best_cfg, best_folds = -1e9, None, None
for lr in LRS:
for lamF in LAMFs:
for lamD in lamDs:
try:
mean, folds = eval_cfg_model(cls, lr=lr, λ_D=lamD, λ_F=lamF, max_ep=10)
if mean > best_mean:
best_mean, best_cfg, best_folds = mean, {"lr": lr, "λ_F": lamF, "λ_D": lamD}, folds
except Exception as e:
print(f"[{RUN_METHOD}] seed={seed} cfg={{'lr':{lr},'λ_F':{lamF},'λ_D':{lamD}}} ERROR: {e}")
if best_cfg is None:
print(f"[{RUN_METHOD}] seed {seed}: no successful runs.")
continue
rec = {"method": RUN_METHOD, "seed": seed,
"cfg": best_cfg, "mean": float(best_mean),
"folds": [float(x) for x in best_folds]}
with open(FOLDS_JL, "a") as f:
f.write(json.dumps(rec) + "\n")
print(f"[{RUN_METHOD}] seed {seed} best {best_cfg} → mean={best_mean:.3f}, folds={best_folds}")
# %%
import os, json, numpy as np, time
SEEDS = list(range(5)) # or 10
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
METHODS = {
"NoDA": (DANNSystem, [0.0]),
"DANN": (DANNSystem, [0.2, 0.4, 0.6]),
"CDAN": (CDAN_Uplift, [0.2, 0.4, 0.6]),
"CORAL": (CORAL_Uplift,[0.2, 0.4, 0.6]),
"MMD": (MMD_Uplift, [0.2, 0.4, 0.6]),
}
RUN_METHOD = "CDAN" # <<< change and rerun per method
cls, lamDs = METHODS[RUN_METHOD]
FOLDS_JL = f"/kaggle/working/folds_{RUN_METHOD}.jsonl"
done_seeds = set()
if os.path.exists(FOLDS_JL):
with open(FOLDS_JL) as f:
for line in f:
obj = json.loads(line)
done_seeds.add(obj["seed"])
for seed in SEEDS:
if seed in done_seeds:
print(f"[{RUN_METHOD}] seed {seed} already done; skipping.")
continue
best_mean, best_cfg, best_folds = -1e9, None, None
for lr in LRS:
for lamF in LAMFs:
for lamD in lamDs:
try:
mean, folds = eval_cfg_model(cls, lr=lr, λ_D=lamD, λ_F=lamF, max_ep=10)
if mean > best_mean:
best_mean, best_cfg, best_folds = mean, {"lr": lr, "λ_F": lamF, "λ_D": lamD}, folds
except Exception as e:
print(f"[{RUN_METHOD}] seed={seed} cfg={{'lr':{lr},'λ_F':{lamF},'λ_D':{lamD}}} ERROR: {e}")
if best_cfg is None:
print(f"[{RUN_METHOD}] seed {seed}: no successful runs.")
continue
rec = {"method": RUN_METHOD, "seed": seed,
"cfg": best_cfg, "mean": float(best_mean),
"folds": [float(x) for x in best_folds]}
with open(FOLDS_JL, "a") as f:
f.write(json.dumps(rec) + "\n")
print(f"[{RUN_METHOD}] seed {seed} best {best_cfg} → mean={best_mean:.3f}, folds={best_folds}")
# %%
import os, json, numpy as np, time
SEEDS = list(range(5)) # or 10
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
METHODS = {
"NoDA": (DANNSystem, [0.0]),
"DANN": (DANNSystem, [0.2, 0.4, 0.6]),
"CDAN": (CDAN_Uplift, [0.2, 0.4, 0.6]),
"CORAL": (CORAL_Uplift,[0.2, 0.4, 0.6]),
"MMD": (MMD_Uplift, [0.2, 0.4, 0.6]),
}
RUN_METHOD = "CORAL" # <<< change and rerun per method
cls, lamDs = METHODS[RUN_METHOD]
FOLDS_JL = f"/kaggle/working/folds_{RUN_METHOD}.jsonl"
done_seeds = set()
if os.path.exists(FOLDS_JL):
with open(FOLDS_JL) as f:
for line in f:
obj = json.loads(line)
done_seeds.add(obj["seed"])
for seed in SEEDS:
if seed in done_seeds:
print(f"[{RUN_METHOD}] seed {seed} already done; skipping.")
continue
best_mean, best_cfg, best_folds = -1e9, None, None
for lr in LRS:
for lamF in LAMFs:
for lamD in lamDs:
try:
mean, folds = eval_cfg_model(cls, lr=lr, λ_D=lamD, λ_F=lamF, max_ep=10)
if mean > best_mean:
best_mean, best_cfg, best_folds = mean, {"lr": lr, "λ_F": lamF, "λ_D": lamD}, folds
except Exception as e:
print(f"[{RUN_METHOD}] seed={seed} cfg={{'lr':{lr},'λ_F':{lamF},'λ_D':{lamD}}} ERROR: {e}")
if best_cfg is None:
print(f"[{RUN_METHOD}] seed {seed}: no successful runs.")
continue
rec = {"method": RUN_METHOD, "seed": seed,
"cfg": best_cfg, "mean": float(best_mean),
"folds": [float(x) for x in best_folds]}
with open(FOLDS_JL, "a") as f:
f.write(json.dumps(rec) + "\n")
print(f"[{RUN_METHOD}] seed {seed} best {best_cfg} → mean={best_mean:.3f}, folds={best_folds}")
# %%
import os, json, numpy as np, time
SEEDS = list(range(5)) # or 10
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
METHODS = {
"NoDA": (DANNSystem, [0.0]),
"DANN": (DANNSystem, [0.2, 0.4, 0.6]),
"CDAN": (CDAN_Uplift, [0.2, 0.4, 0.6]),
"CORAL": (CORAL_Uplift,[0.2, 0.4, 0.6]),
"MMD": (MMD_Uplift, [0.2, 0.4, 0.6]),
}
RUN_METHOD = "MMD" # <<< change and rerun per method
cls, lamDs = METHODS[RUN_METHOD]
FOLDS_JL = f"/kaggle/working/folds_{RUN_METHOD}.jsonl"
done_seeds = set()
if os.path.exists(FOLDS_JL):
with open(FOLDS_JL) as f:
for line in f:
obj = json.loads(line)
done_seeds.add(obj["seed"])
for seed in SEEDS:
if seed in done_seeds:
print(f"[{RUN_METHOD}] seed {seed} already done; skipping.")
continue
best_mean, best_cfg, best_folds = -1e9, None, None
for lr in LRS:
for lamF in LAMFs:
for lamD in lamDs:
try:
mean, folds = eval_cfg_model(cls, lr=lr, λ_D=lamD, λ_F=lamF, max_ep=10)
if mean > best_mean:
best_mean, best_cfg, best_folds = mean, {"lr": lr, "λ_F": lamF, "λ_D": lamD}, folds
except Exception as e:
print(f"[{RUN_METHOD}] seed={seed} cfg={{'lr':{lr},'λ_F':{lamF},'λ_D':{lamD}}} ERROR: {e}")
if best_cfg is None:
print(f"[{RUN_METHOD}] seed {seed}: no successful runs.")
continue
rec = {"method": RUN_METHOD, "seed": seed,
"cfg": best_cfg, "mean": float(best_mean),
"folds": [float(x) for x in best_folds]}
with open(FOLDS_JL, "a") as f:
f.write(json.dumps(rec) + "\n")
print(f"[{RUN_METHOD}] seed {seed} best {best_cfg} → mean={best_mean:.3f}, folds={best_folds}")
# %%
import os, json, numpy as np, time
SEEDS = list(range(5)) # or 10
LRS = [1e-4, 2e-4]
LAMFs = [0, 1, 3]
METHODS = {
"NoDA": (DANNSystem, [0.0]),
"DANN": (DANNSystem, [0.2, 0.4, 0.6]),
"CDAN": (CDAN_Uplift, [0.2, 0.4, 0.6]),
"CORAL": (CORAL_Uplift,[0.2, 0.4, 0.6]),
"MMD": (MMD_Uplift, [0.2, 0.4, 0.6]),
}
RUN_METHOD = "NoDA" # <<< change and rerun per method
cls, lamDs = METHODS[RUN_METHOD]
FOLDS_JL = f"/kaggle/working/folds_{RUN_METHOD}.jsonl"
done_seeds = set()
if os.path.exists(FOLDS_JL):
with open(FOLDS_JL) as f:
for line in f:
obj = json.loads(line)
done_seeds.add(obj["seed"])
for seed in SEEDS:
if seed in done_seeds:
print(f"[{RUN_METHOD}] seed {seed} already done; skipping.")
continue
best_mean, best_cfg, best_folds = -1e9, None, None
for lr in LRS:
for lamF in LAMFs:
for lamD in lamDs:
try:
mean, folds = eval_cfg_model(cls, lr=lr, λ_D=lamD, λ_F=lamF, max_ep=10)
if mean > best_mean:
best_mean, best_cfg, best_folds = mean, {"lr": lr, "λ_F": lamF, "λ_D": lamD}, folds
except Exception as e:
print(f"[{RUN_METHOD}] seed={seed} cfg={{'lr':{lr},'λ_F':{lamF},'λ_D':{lamD}}} ERROR: {e}")
if best_cfg is None:
print(f"[{RUN_METHOD}] seed {seed}: no successful runs.")
continue
rec = {"method": RUN_METHOD, "seed": seed,
"cfg": best_cfg, "mean": float(best_mean),
"folds": [float(x) for x in best_folds]}
with open(FOLDS_JL, "a") as f:
f.write(json.dumps(rec) + "\n")
print(f"[{RUN_METHOD}] seed {seed} best {best_cfg} → mean={best_mean:.3f}, folds={best_folds}")
# %%
import json, numpy as np
from scipy.stats import wilcoxon
def load_folds(method):
folds = []
with open(f"/kaggle/input/folds-results/folds_{method}.jsonl") as f:
for line in f:
obj = json.loads(line)
folds.extend(obj["folds"]) # 3 folds per seed
return folds
METHODS_ORDER = ["NoDA","DANN","CDAN","CORAL","MMD"]
records = {m: load_folds(m) for m in METHODS_ORDER}
def mean_ci(xs):
xs = np.array(xs); mu = xs.mean()
se = xs.std(ddof=1)/np.sqrt(len(xs)) if len(xs)>1 else 0.0
return mu, 1.96*se, xs.std(ddof=1) if len(xs)>1 else 0.0
for m in METHODS_ORDER:
mu, ci, sd = mean_ci(records[m])
print(f"{m:5s} mean={mu:+.3f} ±{ci:.3f} (sd={sd:.3f}) n={len(records[m])}")
def paired(a, b):
n = min(len(a), len(b))
return wilcoxon(np.array(a[:n]), np.array(b[:n]), zero_method="zsplit", correction=True).pvalue
for comp in [("DANN","NoDA"), ("CDAN","DANN"), ("CORAL","DANN"), ("MMD","DANN")]:
print(f"{comp[0]} vs {comp[1]} p={paired(records[comp[0]], records[comp[1]]):.4f}")
# %%
import json, numpy as np, pandas as pd
from pathlib import Path
def load_folds(method):
path = Path(f"/kaggle/input/folds-results/folds_{method}.jsonl")
rows = []
with path.open() as f:
for line in f:
o = json.loads(line)
for i,auuc in enumerate(o["folds"]):
rows.append({"method": method, "seed": o["seed"], "fold": i, "auuc": float(auuc)})
return pd.DataFrame(rows)
methods = ["NoDA","DANN","CDAN","CORAL","MMD"]
dfs = [load_folds(m) for m in methods]
df = pd.concat(dfs, ignore_index=True)
# Make paired table for DANN vs NoDA
pivot = df.pivot_table(index=["seed","fold"], columns="method", values="auuc")
paired = pivot.dropna(subset=["NoDA","DANN"]).reset_index()
paired["diff"] = paired["DANN"] - paired["NoDA"]
paired.head()
# %%
import matplotlib.pyplot as plt
import numpy as np
# Define method order and colors
methods = ["NoDA", "DANN", "CDAN", "CORAL", "MMD"]
colors = ["#4e79a7", "#e15759", "#76b7b2", "#f28e2b", "#59a14f"]
# Prepare data
data = [df[df.method == m]["auuc"].values for m in methods]
# Create figure
plt.figure(figsize=(5, 5))
# Violin plot
vp = plt.violinplot(data, showmeans=True, showextrema=False)
# Color violins
for i, pc in enumerate(vp['bodies']):
pc.set_facecolor(colors[i])
pc.set_edgecolor("black")
pc.set_alpha(0.7)
# Boxplot overlay
plt.boxplot(data, positions=range(1, len(methods) + 1), widths=0.2, patch_artist=True,
boxprops=dict(facecolor='white', color='black'),
medianprops=dict(color='black'))
# Jittered individual points
for i, y in enumerate(data):
x = np.random.normal(i + 1, 0.04, size=len(y))
plt.plot(x, y, 'k.', alpha=0.3)
# Axis labels and title
plt.xticks(range(1, len(methods) + 1), methods)
plt.xticks(fontsize=12) # x-axis numbers
plt.yticks(fontsize=12) # y-axis numbers
plt.xlabel("Model Type",fontfamily='serif', fontsize=16)
plt.ylabel("AUUC",fontfamily='serif', fontsize=16)
#plt.title("Distribution of LOSO-AUUC by Model Type")
# plt.grid(linestyle=':', linewidth=0.75, color='black')
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
plt.tight_layout()
# Save outputs
plt.savefig("/kaggle/working/auuc_violin_by_method.png", dpi=500)
plt.savefig("/kaggle/working/auuc_violin_by_method.pdf", dpi=500)
plt.show()
# %%
plt.figure(figsize=(6,4))
plt.hist(paired["diff"], bins=10, edgecolor="k", alpha=0.8)
# plt.grid(linestyle=':', linewidth=0.85, color='black')
plt.grid(True, linestyle=(0, (1, 5)), linewidth=1.2, color='black')
plt.xticks(fontsize=12) # x-axis numbers
plt.yticks(fontsize=12) # y-axis numbers
plt.xlabel("AUUC(DANN) − AUUC(NoDA)",fontfamily='serif', fontsize=16)
plt.ylabel("Count",fontfamily='serif', fontsize=16)
# plt.title("Distribution of paired differences")
plt.tight_layout()
plt.savefig("/kaggle/working/diff_hist_DANN_NoDA.png", dpi=600)
plt.savefig("/kaggle/working/diff_hist_DANN_NoDA.pdf", dpi=600)
plt.show()
# %%
import numpy as np
def bootstrap_ci(arr, iters=2000, func=np.median, alpha=0.05):
arr = np.array(arr)
boots = [func(np.random.choice(arr, size=len(arr), replace=True)) for _ in range(iters)]
lo, hi = np.percentile(boots, [100*alpha/2, 100*(1-alpha/2)])
return func(arr), lo, hi
def cliffs_delta(a, b):
a = np.array(a); b = np.array(b)
gt = sum(x>y for x in a for y in b)
lt = sum(x<y for x in a for y in b)
n = len(a)*len(b)
return (gt - lt)/n
median, lo, hi = bootstrap_ci(paired["diff"])
delta = cliffs_delta(paired["DANN"].values, paired["NoDA"].values)
print(f"Median paired diff={median:.3f} [{lo:.3f},{hi:.3f}] Cliff's Δ={delta:.3f}")

dann-analysis.ipynb at commit 2f5e847, no license · at the source

Overview

Authors: Abdullah Nayem Wasi Emran^1,2, A. B. M. Alim Al Islam¹

ORCID iDs: Abdullah Nayem Wasi Emran

Institutions: Bangladesh University of Engineering and Technology (Bangladesh); BRAC University (Bangladesh)

Journal: iScience, volume 29, issue 6, article 116027

Dates: received 18 February 2026; accepted 4 May 2026; published online 22 May 2026

Type: Research article · Language: English

License: CC BY

Identifiers: DOI 10.1016/j.isci.2026.116027 · PMID 42231948 · PMCID PMC13224034 · OpenAlex W7162112417

Open access: gold, a free copy (OpenAlex)

Status: code verified

Categories: human (organism), clinical / translational (subfield)

Methods: Connectivity, Statistics, Machine learning

Keywords: Neuroscience, Machine learning, Psychology

Topic: Mental Health Research Topics (Experimental and Cognitive Psychology, Psychology), according to OpenAlex

Funding: BUET

Citations: not cited yet (Europe PMC); 69 references in the paper

Abstract

Predicting who will deteriorate under stress is important for targeting mental-health support; yet, treatment-effect models are rarely tested across populations. We evaluate a domain-adaptive neural uplift model on three heterogeneous cohorts—medical students, members of the general public under quarantine, and psychiatric patients (n = 2,624). The model combines a shared encoder, two potential-outcome heads, a domain discriminator, and an optional fairness penalty. We compare no-adaptation training with four domain-adaptation mechanisms under a leave-one-domain-out protocol, using AUUC and a semi-synthetic benchmark with known treatment effects. The model achieves positive uplift ranking in two cohorts, while the psychiatric cohort shows sign inversion of the effect proxy. Adaptation yields modest, tuning-sensitive gains over a strong baseline. These results clarify when domain adaptation helps treatment-effect ranking under distribution shift and inform cautious deployment across cohorts.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 10 matches between paragraphs and lines of code.

Nayem9274/domain-adaptive-uplift-mh

License: none: the authors keep all their rights

State: the link answers, verified on 28 September 2026

Evidence: files inventoried

Commit: 2f5e847a6f246880c9e33699351f8f3c3ca63f38, 14 April 2026

Languages: Jupyter (1)

Size: 3 files, 1 script

Software Heritage: not archived

Found in: “Data and code availability”

Holds: README, environment (requirements.txt), 1 notebook

Not found: license file, CITATION.cff, tests, continuous integration, documentation

Tools: PyTorch Lightning (1 file), Matplotlib (1 file), NumPy (1 file), pandas (1 file), PyTorch (1 file), scikit-learn (1 file), SciPy (1 file), seaborn (1 file), statsmodels (1 file)

Availability: 1 check, the latest on 28 September 2026: the link answers

28 September 2026: the link answers

2 files

dann-analysis.ipynb, Jupyter, 2,286 lines, 10 matches
README.md, Text, 9 lines

At the source: github.com/Nayem9274/domain-adaptive-uplift-mh

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
1 script, each with its path and the digest of its content;
10 matches between paragraphs of the paper and lines of the code (method lexical-v1);
neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

data.mendeley.com/datasets/pxjmjyfdh2, at Mendeley Data; found in “Data and code availability”
osf:8bsvr, at OSF; found in “Data and code availability”
zenodo:5702895, at Zenodo; found in “Data and code availability”

Data and code availability

• All datasets analyzed in this study are publicly available from third-party repositories and were not generated by the authors. The Identification of Major Psychiatric Disorders Dataset is available at https://osf.io/8bsvr/. The Medical Students Mental Health Dataset is available at https://zenodo.org/records/5702895. The Mental Health Depression During Quarantine Life Dataset is available at https://data.mendeley.com/datasets/pxjmjyfdh2/1. All datasets are openly accessible subject to the terms of their respective repositories. • The code generated during this study is publicly available at https://github.com/Nayem9274/domain-adaptive-uplift-mh. The repository contains the analysis notebook and a requirements.txt specifying all software dependencies needed to reproduce the reported experiments. • No new datasets were generated or deposited as part of this study. This study does not report original data.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 28 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 3 keywords, 1 funder, 41 references.

Cite

This paper

Emran, A. N. W., & Al Islam, A. B. M. A. (2026). Domain adaptive uplift modeling across heterogeneous mental health cohorts. iScience, 29(6), 116027. https://doi.org/10.1016/j.isci.2026.116027

BibTeX

@article{emran2026domain, author = {Emran, Abdullah Nayem Wasi and Al Islam, A. B. M. Alim}, title = {{Domain adaptive uplift modeling across heterogeneous mental health cohorts}}, journal = {iScience}, year = {2026}, month = may, volume = {29}, number = {6}, pages = {116027}, publisher = {Elsevier}, issn = {2589-0042}, doi = {10.1016/j.isci.2026.116027}, url = {https://doi.org/10.1016/j.isci.2026.116027}, pmid = {42231948}, pmcid = {PMC13224034} }

RIS

TY - JOUR AU - Emran, Abdullah Nayem Wasi AU - Al Islam, A. B. M. Alim TI - Domain adaptive uplift modeling across heterogeneous mental health cohorts T2 - iScience J2 - iScience PY - 2026 DA - 2026/05/22 VL - 29 IS - 6 SP - 116027 SN - 2589-0042 PB - Elsevier DO - 10.1016/j.isci.2026.116027 UR - https://doi.org/10.1016/j.isci.2026.116027 LA - en ER -

CSL-JSON

{ "id": "10.1016/j.isci.2026.116027", "type": "article-journal", "title": "Domain adaptive uplift modeling across heterogeneous mental health cohorts", "container-title": "iScience", "author": [ { "family": "Emran", "given": "Abdullah Nayem Wasi" }, { "family": "Al Islam", "given": "A. B. M. Alim" } ], "container-title-short": "iScience", "volume": "29", "issue": "6", "page": "116027", "DOI": "10.1016/j.isci.2026.116027", "PMID": "42231948", "PMCID": "PMC13224034", "ISSN": "2589-0042", "publisher": "Elsevier", "URL": "https://doi.org/10.1016/j.isci.2026.116027", "language": "en", "issued": { "date-parts": [ [ 2026, 5, 22 ] ] } }

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.