Robust Real-Time Monocular Human Pose Estimation with Insole-Based Sensor Fusion
Real-time monocular human pose estimation breaks down when the lower body is occluded or the subject steps out of frame. This paper introduces a part-decoupled temporal fusion framework that augments monocular video with sparse plantar-pressure and foot-mounted acceleration from smart insoles. The core insight is that cheap, wearable foot sensors provide complementary physical constraints exactly where monocular vision is weakest, enabling robust pose and gait recovery under severe visual degradation.
The paper makes a strong practical case for unobtrusive foot-sensor augmentation in real-time pose estimation. Its part-decoupled architecture is physically motivated and empirically effective, yielding large accuracy and smoothness gains under synthetic and real occlusion across three diverse visual backbones. However, the evaluation has significant gaps: no vision-only baseline is reported under complete visual dropout, cross-subject generalization is shown only for MediaPipe, and the method is limited to root-relative pose without global translation. These omissions temper the claim that the system is fully ready for real-world deployment.
The strongest aspect is the experimental demonstration that four-point pressure plus a single accelerometer meaningfully improves lower-body pose across clean, occluded, and dropout conditions. The part-decoupled design is well justified: the ablation shows that naive temporal fusion is inferior to the proposed upper/lower stream separation, and removing the global coordinator degrades whole-body consistency. The mobile runtime benchmark is also compelling—the fusion module adds less than 1.5 ms on a mid-range phone, confirming it is deployable.
First, the paper omits vision-only baselines for the complete visual-dropout condition, arguing that they cannot serve as a meaningful comparative baseline. Without these numbers, it is impossible to quantify how much of the dropout performance is due to insole sensing versus temporal state memorization.
Second, cross-subject generalization is tested only with the MediaPipe backbone and on standard actions, leaving open whether the gains transfer to SMPL-based backbones and unseen occlusion scenarios. Third, the output is strictly root-relative, so the method cannot recover global foot placement or long-horizon trajectories, which limits utility in open-world motion capture.
The evidence clearly supports the claim that foot sensors improve monocular pose under lower-body occlusion: Table II shows MPJPE drops by roughly 30–40% for MediaPipe and YOLOPose26, and Table V confirms this on a physically occluded desk sequence. Gait metrics in Table III further strengthen the story by showing improved stance accuracy and cadence. Comparisons to related work are fair but largely qualitative; the paper does not benchmark against SolePoser or PressInPose on shared data, making it hard to judge whether the gains come from the fusion strategy or simply from having a camera present in addition to insoles.
The paper describes the network architecture, loss weights ($\lambda_v=0.1$, $\lambda_a=0.4$, $\lambda_b=0.02$), and training window size ($T=120$), but omits the learning rate, batch size, total epochs, and exact train/validation split ratios. The custom 890K-frame dataset and preprocessing scripts are promised for release, which is critical because the dataset is not publicly available elsewhere. The fusion module's 12.10 MB ONNX model and sub-1.5 ms runtime are specified, aiding mobile reproduction. Until the dataset and code are actually released, independent reproduction is blocked.
Real-time monocular human pose estimation is an important sensing component for human-centered intelligent systems. While recent camera-based methods have achieved impressive accuracy, fundamental challenges persist, particularly when visual observations are incomplete, such as during lower-body occlusion or short out-of-frame intervals. In these challenging but common scenarios, vision-only systems often struggle to reconstruct physically plausible lower-limb motion and reliable foot-ground contact. Due to the highly unobtrusive and wearable nature of smart insoles, this paper investigates whether simple plantar pressure and acceleration sensors can effectively resolve these visually induced limitations. To systematically investigate this, we first collect a synchronized multimodal dataset and then propose a part-decoupled temporal fusion framework. The proposed model fuses foot-sensing features into the lower-body branch to compensate for incomplete visual evidence, while a lightweight coordinator exchanges information between the upper- and lower-body temporal representations to maintain whole-body consistency. The framework is evaluated with three representative real-time visual backbones based on 2D keypoints, 3D landmarks, and SMPL-based human mesh recovery. We evaluate our framework on three representative real-time visual backbones. Quantitative experiments demonstrate that incorporating simple foot sensors consistently improves pose accuracy, motion smoothness, and gait-related measurements over vision-only baselines, with the most significant gains observed under occlusion and short visual-dropout conditions. Ultimately, our results demonstrate that unobtrusive foot sensing provides vital and complementary physical observability when camera observations are temporarily unavailable.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.