EyeTAG: Eye Trajectory-Aware Gaze Estimation
EyeTAG tackles temporal jitter in video gaze estimation by making gaze motion an explicit variable rather than an implicit deep feature. It feeds back first-order differences of its own recent predictions as a compact kinematic token, then fuses this with face and eye streams via cross-attention and a causal Transformer decoder. The result is a causal streaming model that cuts mean angular error on Gaze360 by roughly a degree and claims subject-invariant motion via translation-invariant differencing.
EyeTAG is a solid, well-motivated contribution to video-based gaze estimation. The core idea—an explicit autoregressive differential prior—is cleanly executed, and the within-model ablations do a good job isolating its effect from generic temporal smoothing. The roughly 1° gain on Gaze360 is meaningful, but the EVE result is effectively a tie (2.56° vs. 2.58°) and the model is an order of magnitude more expensive than the strongest single-frame competitors. The paper is honest about the STAGE baseline mismatch and its desktop-only latency, which keeps the assessment balanced.
The explicit kinematic prior is the strongest selling point. By differentiating past predictions $\Delta\hat{\mathbf{G}}$ and feeding them through a lightweight MLP, the model turns temporal coherence from a hidden inductive bias into a directly supervised quantity. The within-model ablations are carefully controlled: they keep the encoder and fusion architecture fixed and vary only the gaze-history component, making it possible to attribute the near-zero saccade bias to the differential formulation itself. The temporal stability analysis—fixation jitter, saccade bias, and per-frame MAE traces—goes well beyond a single aggregate error and supports the claim that the benefit is not just smoothing. Finally, releasing code and specifying hyperparameters aids reproducibility.
The first concern is geometric: the differential prior operates on raw 3D gaze vectors in $\mathbb{R}^3$, not on the angular manifold $\mathcal{S}^2$. Subtracting unit vectors gives a chord-length proxy, not a true angular velocity, so the claim that differencing yields subject-invariant motion is heuristic rather than formally grounded on the sphere. Second, the EVE improvement is marginal (0.02°) and no statistical significance tests are reported, so 'on par' is the only defensible reading. Third, latency and cost are high: the authors note that EyeTAG needs roughly 20 ms per $T=48$ window on an RTX A6000, but the model consumes 121.57 GFLOPs, compared with 1.43 GFLOPs for CrossGaze. This is acknowledged as desktop feasibility only, yet the deployment gap remains large. Fourth, the source of face and eye crops is never described; if an external detector or landmarks are used, reproducibility hinges on an unspecified preprocessing pipeline.
Gaze360 evidence is convincing: a backbone-controlled check shows that swapping the face encoder to the same VGGFace2 ResNet-50 improves the strongest multi-frame baseline to 9.76°, while EyeTAG stays at 8.83°, indicating the gain comes from the fusion and prior, not pretraining alone. The ablation in Table 5 directly supports the differential representation over absolute history. The comparison to STAGE, however, is weakened by the authors' own caveat that STAGE was designed for personalized estimation and is being evaluated calibration-free. That caveat is commendably transparent, but it means the multi-frame field on Gaze360 is thinner than the table suggests. The temporal metrics (Table 3) are a welcome addition, though they would be stronger with confidence intervals or paired statistical tests.
Training hyperparameters are thorough: $T=48$, AdamW with $\beta_1=0.9$, $\beta_2=0.999$, weight decay $2\times10^{-2}$, peak learning rate $3\times10^{-4}$ with cosine annealing over 15 epochs, batch size 32, gradient clipping at 1.0, and a 4-frame sampling stride. Scheduled sampling details and the padding strategy for cold-start frames are also spelled out. The authors provide a GitHub link for code. What is missing is any description of how face and eye crops are generated—detector, landmarks, or heuristics—which is a critical pipeline step. EVE evaluation uses the public validation split because test labels are unavailable; this is acceptable but limits comparability to prior work that may report test numbers. Latency is reported only on a high-end desktop GPU, so embedded reproducibility is unclear. No random seed or multiple-run variance is mentioned.
Gaze estimation under natural head-eye motion underpins applications from driver monitoring to human-computer interaction. Single-frame methods predict each frame independently, so consecutive outputs fluctuate as jitter. Multi-frame methods reduce this, but they learn motion implicitly inside appearance features, so the gaze trajectory is never an explicit variable. We propose EyeTAG (Eye Trajectory-Aware Gaze Estimation), a causal multi-frame framework built around an explicit first-order gaze prior: at each step it differentiates its own recent predictions and feeds the resulting trajectory back as a compact kinematic token. Because differencing is translation-invariant in gaze space, this token carries subject-invariant motion rather than personal gaze offsets. Face and eye streams supply visual evidence, fused by cross-attention and a causal Transformer decoder. EyeTAG reduces the mean angular error by about 1.0$^\circ$ on Gaze360 and performs on par with the strongest baseline on EVE (2.56$^\circ$ vs. 2.58$^\circ$). Within-model ablations, which keep the encoder and the rest of the architecture fixed and vary only the gaze history, show that the differential formulation, rather than temporal context alone, removes the systematic saccade bias that persists even with an absolute gaze-history prior. Our code is available at https://github.com/peter8366/EyeTAG.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.