W-JEPA: Wavelet Joint-Embedding Predictive Architecture for Wearable Activity Recognition
Wearable activity recognition suffers from sensor heterogeneity and subtle, scale-specific motion patterns that models often confuse with participant-specific artifacts. W-JEPA tackles this by replacing raw temporal patches with fixed wavelet tokens—indexed by physical scale, sensor axis, and time—as the units of masked latent prediction in a Joint-Embedding Predictive Architecture. The key payoff is improved generalization to unseen participants across six benchmarks.
The paper makes a strong empirical case that wavelet-based tokenization improves subject-disjoint generalization over raw-patch JEPA and other SSL baselines on wearable activity recognition. Across six datasets, W-JEPA achieves the highest mean accuracy and macro-F1 under participant holdout, and the controlled tokenizer ablation shows the wavelet grid accounts for most of this margin. Yet the gains are uneven: on USC-HAD the wavelet variant barely edges the patch-based HAR-JEPA baseline, and W-JEPA without wavelets underperforms it, suggesting the backbone architecture is not uniformly superior and the tokenizer is doing the heavy lifting.
The controlled ablation isolating the tokenizer is the strongest element. By holding the JEPA pipeline fixed and swapping wavelet tokens for raw temporal patches, the authors show that wavelets contribute disproportionately when test participants are unseen, adding roughly seven points of macro-F1 over the no-wavelet variant. The exercise-bike example provides an intuitive mechanistic explanation: scale-specific oscillations visible only in detail bands resolve near-identical raw signals, and the fixed wavelet grid carries the same physical meaning across sensor setups because all recordings are resampled to a common rate.
The gains are uneven and the paper underplays cases where wavelets help little. The authors acknowledge that RealWorld shows smaller and less consistent gains, but they do not diagnose why. On USC-HAD, W-JEPA without wavelets actually underperforms HAR-JEPA, suggesting the backbone architecture itself may be weaker than the baseline when the tokenizer is removed. The attention-under-noise analysis is descriptive yet explicitly stops short of causal claims, and the choice to use only a variance hinge—dropping VICReg’s covariance and invariance terms—is justified only by a brief ablation.
The evidence broadly supports the central claim because the authors re-ran all eight baselines with identical preprocessing, splits, and seeds, reducing implementation bias. However, HAR-JEPA was originally developed for the REALDISP dataset, so its re-implementation on these six benchmarks may not reflect its best-tuned performance. The wavelet tokenizer is clearly the driver of improvement, because W-JEPA without it trails HAR-JEPA on USC-HAD and is nearly identical to it on UCI-HAR. It remains unclear whether non-JEPA baselines were given comparable architectural capacity, since W-JEPA uses an eight-layer transformer of width 384.
Reproduction looks feasible. The authors provide source code, pretrained weights, and dataset splits. Hyperparameters are specified in Appendix B: $P=2$ bins per band, mask ratio $\rho=0.65$, 50 pretraining epochs at batch size 256, 70 fine-tuning epochs at batch size 64, AdamW with weight decay 0.05, and a cosine learning-rate schedule. What is missing is an estimate of total compute and a broader variance analysis beyond five seeds, which is especially modest given that some subject-disjoint splits rely on only two or three training participants.
A major challenge in wearable device activity recognition is the heterogeneity across datasets in sensor placement, channel count, sampling rate and subjects. Moreover, the motion patterns that distinguish activities often appear as subtle, scale-specific variations within otherwise similar waveforms. As a result, raw signals are often ambiguous, and models trained on them tend to rely on cues specific to a particular sensor setup or participant, which limits both cross-dataset transfer and generalization to unseen participants. Addressing this requires representations that preserve such activity-relevant, scale-specific variation. We introduce W-JEPA, a self-supervised framework that exploits the multiscale structure of inertial motion and leverages a joint-embedding predictive architecture, by organizing windows into a fixed grid of wavelet tokens indexed by scale, sensor axis, and time. These wavelet tokens help expose activity-specific detail within broadly similar raw signals, and serve as the units of masked latent prediction. W-JEPA predicts masked token representations supplied by an exponential-moving-average (EMA) target encoder. Because recovering a masked token requires relating motion at one scale and axis to motion at other scales and axes, the model learns the cross-scale, cross-axis structure of human movement. Across six wearable activity benchmarks, W-JEPA outperforms state-of-the-art methods under random-window splits and remains the strongest when subjects are held out, improving average macro-F1 by 8.7 percentage points over the strongest published baseline. These results indicate that physically grounded multiscale tokenization, combined together with latent predictive learning, provides a promising foundation for transferable representations of heterogeneous wearable inertial data.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.