Artifact Annotations Partially Substitute for Per-User Calibration: SAFE-EDA and a Normalization-Controlled Evaluation of Wrist-EDA Affect Recognition
Wrist electrodermal activity differs in amplitude from person to person, so affect-recognition models normalize the signal before classification. A hidden protocol choice is whether the normalization statistics come from training subjects only or from the held-out user's own complete recording, which retroactively gives the model a calibration that a newly worn device lacks. The paper introduces SAFE-EDA, a compact convolutional network pretrained on expert artifact annotations, and shows that its measured benefit over training from scratch ranges from 0.020 to 0.227 depending on this normalization choice, with the larger gains occurring in the realistic first-use setting. The authors also survey fifty published WESAD studies and find only ten percent report the source of their normalization statistics, making published scores difficult to compare.
The paper makes a valuable and well-controlled contribution. By fixing the architecture, folds, and training budget while varying only initialization and normalization source, it isolates how a preprocessing choice alters the apparent value of pretraining. The literature survey is sobering, the factorial design and paired subject-level inference are sound, and the extensive ablations lend credibility. However, the interaction direction reverses on the Wearable dataset, which undermines the universality of the partial-substitution claim and points to dataset-specific mechanisms that remain unexplained.
The reporting survey is concrete and actionable: among 50 WESAD studies, window length was reported in 52% but the normalization source in only 10%, with a 95% Wilson interval of 4–21%. The experimental controls are exemplary. Artifact supervision proved far more useful than self-supervised pretraining on identical recordings (+0.078 versus +0.008 macro-F1), and shuffled-label pretraining captured only about 58% of the full gain, showing that the expert label content matters. The subject-level paired design—average seeds within each participant before Wilcoxon signed-rank tests—correctly treats overlapping windows as correlated observations rather than independent units.
The partial-substitution story—that per-user normalization and pretraining supply redundant amplitude information—holds on WESAD but breaks on the Wearable dataset, where per-user normalization enlarged rather than shrank the transfer gain. The authors acknowledge that 'the direction of the interaction depends on the data,' yet they do not identify whether task differences, amplitude distributions, or preprocessing explain the reversal. Additionally, the fixed 12-epoch budget left sparse-hop training losses high (0.871 scratch versus 0.524 transfer at hop 60 under global-train), suggesting that some of the reported gain reflects faster convergence rather than superior final capacity. Several controls—classical baselines, shuffled-label tests, and architectural ablations—were run only at the dense-window setting, leaving the large sparse-hop gains less decomposed than they appear.
The evidence supports the core claim that protocol choices can dominate architecture choices: the same pretrained network gained 0.227 at global-train hop 60 but only 0.020 under per-subject-z, while scratch performance swung by 0.135 across hops under global-train but only 0.057 under per-subject-z. Comparisons to prior work are appropriately scoped; the authors note that their wrist-only accuracy of 0.7815 under per-user normalization overlaps the original multimodal chest-sensing benchmark, rather than claiming superiority. The shuffled-label control effectively separates the contribution of supervised pretraining as such from the content of the labels. The caveat is that the Wearable reversal limits how strongly one can generalize the specific interaction direction, though the broader message—that interaction exists and reporting matters—remains intact.
Reproducibility is strong on protocol transparency but has practical limits. The authors report every cell of the WESAD grid without selection, release code on GitHub and Zenodo, and specify a fixed AdamW learning rate of 0.001 across 12 epochs with no early stopping. However, the fixed epoch budget means sparse-hop models were substantially unconverged, and an independent reproduction would need to decide whether to match the fixed budget or train to convergence. Two Wearable identifiers were missing from the authors' processed copy, and the literature survey was coded by a single author without independent verification. The decomposition branch was mostly inactive, and several ablations were restricted to the dense-window protocol, so reproducing the full result table requires both exact software versions and adherence to an arguably suboptimal training stopping rule.
Wrist electrodermal activity (EDA) differs in amplitude from one person to the next, so affect-recognition models normalize their input before classification. Studies that test such models on held-out subjects seldom report where the normalization statistics come from, yet statistics computed from the held-out subject's own recording give the model information that a device does not have when it is first worn. We asked how this choice alters the measured benefit of pretraining. A compact convolutional network, SAFE-EDA, was pretrained on expert artifact annotations from 43 subjects and compared with the same network trained from scratch on the Wearable Stress and Affect Detection (WESAD) dataset (15 subjects, leave-one-subject-out), with two normalization sources crossed with four window hops. When the statistics came only from training subjects, pretraining raised macro-F1 by 0.078 to 0.227; when they came from the held-out user's full recording, the gain fell to between 0.020 and 0.050 and was no longer significant. Artifact supervision was far more useful than self-supervised pretraining on the same recordings (0.078 versus 0.008). Across 13 configurations in two datasets, the pretrained network was better in 12, but on the second dataset (26 subjects) per-user normalization increased the gain instead of reducing it, so the interaction depends on the data. Only five of 50 published WESAD studies state which data were used for normalization. Reporting this choice is necessary to separate first-use performance from performance after calibration.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.