Towards Field-Deployable Wearable IMU Systems for Occupational Activity Recognition

cs.LG Ihsan Ul Haq (1, 2), James Brusey (1), Farnad Nasirzadeh (2), Chandan Karmakar (3) ((1) Centre for Computational Science, Mathematical Modelling, Coventry University, United Kingdom, (2) School of Architecture, Built Environment Deakin University, Australia, (3) School of Information Technology Deakin University, Australia) · Oct 22, 2026
Local to this browser
What it does
The paper tackles occupational activity recognition for construction workers using wearable IMUs. It shows that headline ~99% accuracies on the VTT-ConIoT benchmark collapse to ~77% mean under strict leave-one-subject-out evaluation, with...
Why it matters
It shows that headline ~99% accuracies on the VTT-ConIoT benchmark collapse to ~77% mean under strict leave-one-subject-out evaluation, with a 27pp per-subject spread. The work systematically maps how sensor placement, window length,...
Main concern
The paper makes a valuable methodological contribution by enforcing rigorous LOSO evaluation and exposing the gap between aggregate lab headlines and individual-level reality. The per-subject decomposition is genuinely useful, and the...
Community signal
0
0 up · 0 down
Sign in to vote with arrows
AI Review AI reviewed
Plain-language introduction

The paper tackles occupational activity recognition for construction workers using wearable IMUs. It shows that headline ~99% accuracies on the VTT-ConIoT benchmark collapse to ~77% mean under strict leave-one-subject-out evaluation, with a 27pp per-subject spread. The work systematically maps how sensor placement, window length, classifier choice, augmentation, and personalised calibration affect true subject-independent performance.

Critical review
Verdict
Bottom line

The paper makes a valuable methodological contribution by enforcing rigorous LOSO evaluation and exposing the gap between aggregate lab headlines and individual-level reality. The per-subject decomposition is genuinely useful, and the CTGAN failure analysis is well-explained. However, the work is limited by a small, homogeneous, lab-collected dataset and some interpretive overreach: the title promises field-deployable systems, but no field data are collected or validated.

“Using the VTT-ConIoT benchmark, we conduct, to our knowledge, the first systematic configuration sensitivity analysis of this dataset: six classifiers across five sensor placements and three window sizes under a single LOSO protocol”
paper · Abstract
“Individual accuracies range from 61.9 % to 88.7 % (mean 77.5 % ± 8.3 %, macro-F1 75.1 %), a 27 pp spread.”
paper · Section IV-A
What holds up

The core finding that aggregate statistics conceal large per-subject variance is well-supported and important for the HAR literature. The systematic sweep across sensor placements, window sizes, and classifiers under a single consistent protocol provides evidence-grounded guidance. The CTGAN analysis is a particular strength: the authors demonstrate via Kolmogorov-Smirnov testing that synthetic samples are not distributionally faithful and offer a clear mechanism---CTGAN treats each time-step as an independent tabular row, destroying autocorrelation.

“A per-feature Kolmogorov-Smirnov test found 294/294 (100 %) feature comparisons distinguishable from real data at p<0.05: the synthetic samples are not distributionally faithful, consistent with CTGAN treating each time-step as an independent tabular row and losing IMU windows' autocorrelation.”
paper · Section IV-D
“Among the four weakest participants, mean accuracy increased from 68.4 % without calibration data to 85.2 % after adding 30 % of each participant's own temporally disjoint data to the training set, recovering 16.8 pp.”
paper · Section IV-E
Main concerns

The dataset is small ($n=13$, only 3 female) and entirely lab-collected, yet the paper's title and framing promise field-deployable systems without validating any claim outdoors or on a construction site. The CNN baseline is intentionally simple (a from-scratch 1-D CNN), so its poor performance ($55.7$% accuracy) is used to suggest prior ~99% deep-learning results are merely protocol artifacts; however, the authors never re-implement those prior architectures under LOSO, leaving a genuine architecture gap unaddressed. The citation of Leng et al. [12] to support the claim that 'GAN-based augmentation does not reliably outperform standard time-series transforms' appears imprecise: Leng et al. study cross-modality virtual IMU generation (video/text-to-IMU) and report that it significantly improves performance over classical augmentation, which is a different generative paradigm than CTGAN tabular augmentation. Finally, the personalised calibration result is only reported for the four weakest participants, without a corresponding full-cohort control.

“generative adversarial network (GAN)-based augmentation [10] does not reliably outperform standard time-series transforms [11, 12]”
paper · Section II-A
“Results show that virtual IMU data significantly improves performance over real or augmented data alone, particularly under limited-data conditions.”
Evidence and comparison

The evidence strongly supports the central claim that subject-independent evaluation drastically lowers accuracy and reveals hidden variance. The comparisons to prior VTT-ConIoT studies are fair in noting that many use $k$-fold or subject-blind splits rather than LOSO. However, the within-paper comparison between classical classifiers and the deep-learning baseline is weaker: the authors chose a deliberately simple CNN to reflect 'out-of-the-box deployment,' but then use its failure to argue that classical models are 'more sample-efficient.' Without testing the actual architectures from prior work under the same LOSO protocol, this comparison is incomplete. The hyperparameter tuning check (at most $1.1$ pp delta) is a useful robustness control.

“A class-weighted, regularised 1-D CNN achieved only 55.7 % mean accuracy (macro-F1 51.9 %) under the same protocol, clearly below the classical classifiers. Classical feature-based classifiers thus remain more sample-efficient than a from-scratch CNN here, far below the ≈ 99 % deep-learning figures reported elsewhere for this dataset”
paper · Section IV-B
“a nested hyperparameter search found a tuned-vs-default delta of at most 1.1 pp (and for RF, tuning did not outperform defaults at all), indicating defaults are not a materially unfair basis for this classifier comparison.”
paper · Section IV-B
Reproducibility

The authors commit to open science: code and aggregated results are available on GitHub. Most hyperparameters are fixed to scikit-learn defaults with only the random seed set, which improves reproducibility. The preprocessing pipeline is clearly described (147 statistical features, $z$-score standardisation per fold). However, the CTGAN subset selection (6 of 13 folds) and the nested search details (search space, budget) are underspecified. The personalised calibration relies on a 'temporal order' split to avoid window-overlap leakage, but the exact split implementation should be described more precisely. Independent reproduction is feasible for the main experiments, though the small dataset size means variance estimates will remain noisy.

“Code and aggregated results to reproduce the experiments in this paper are available at https://github.com/IhsanUlHaq/bsn2026-imu-har.”
paper · Acknowledgement and Code Availability
“Six classifiers were evaluated: Random Forest (RF), Extra Trees (ET), XGBoost (XGB), Linear Discriminant Analysis (LDA), Logistic Regression (LR), and SVM (RBF kernel), all with default scikit-learn hyperparameters (only the random seed fixed and, where needed for convergence, the iteration limit increased) to reflect out-of-the-box deployment.”
paper · Section III-C
Abstract

Wearable inertial measurement unit (IMU) systems for construction worker human activity recognition (HAR) routinely report aggregate accuracies above 95 %, rarely under genuine subject-independent (leave-one-subject-out (LOSO)) evaluation, masking the per-individual variance that determines real-world deployment performance. Using the VTT-ConIoT benchmark, we conduct, to our knowledge, the first systematic configuration sensitivity analysis of this dataset: six classifiers across five sensor placements and three window sizes under a single LOSO protocol, a personalised-calibration pilot, and a validated comparison of tabular GAN-based and standard time-series augmentation. Per-subject LOSO accuracy for the main multi-sensor, 16-class configuration ranges from 61.9 % to 88.7 % (mean 77.5 %), a 27 pp spread, though all data are laboratory-collected. Multi-sensor fusion and longer windows both improve accuracy; standard augmentation gives small, consistent gains, while Conditional Tabular GAN (CTGAN) augmentation actively harms accuracy with a demonstrated distributional-fidelity mechanism; and personalised calibration recovers 16.8 pp for the weakest participants from just 30 % of their own data. These findings provide evidence-grounded configuration guidance for scaling occupational body sensor networks towards field deployment

Challenge the Review

Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.

No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.