The Impact of Detector Architecture, Model Scale, and Scene Density on Multi-Pedestrian Tracking
This paper tackles the practical problem of choosing detectors and trackers for multi-pedestrian tracking across scenes with different densities. It carries out a broad empirical comparison of the YOLO11, RT-DETR, and YOLO26 detector families paired with ByteTrack and BoT-SORT on MOT17, MOT20, and SOMPT22, including an ablation of NMS versus end-to-end inference for YOLO26. The study offers a useful survey of quality-speed trade-offs, although its conclusions are constrained by out-of-the-box COCO weights and very small benchmark slices.
This is a well-organized empirical survey with a clean, modular protocol and publicly released code. The descriptive rankings are internally consistent, but the paper remains largely observational: with only three sequences per benchmark, no fine-tuning, and no causal ablations of detector internals, the findings are more prescriptive than explanatory.
The paper’s main strength is breadth under a single protocol. By evaluating three detector families at five scales, two trackers, three benchmarks with contrasting densities, and an NMS versus end-to-end ablation, the authors provide concrete guidance: RT-DETR leads on HOTA especially in crowded MOT20, whereas YOLO26-L with NMS delivers a favorable quality-speed trade-off on MOT17 and SOMPT22. The results are presented in dense but clear tables, and the artifact repository is shared.
The evaluation is worryingly thin, with only three sequences per benchmark, leaving substantial sampling uncertainty. Because all models run zero-shot from COCO without any benchmark-specific fine-tuning, the absolute numbers are likely pessimistic and the relative gaps may not reflect standard practice. Additionally, the NMS versus end-to-end ablation is purely descriptive: the authors observe higher fragmentation in end-to-end YOLO26 but do not probe confidence distributions or temporal stability to explain the gap.
The evidence supports the headline rankings descriptively: RT-DETR achieves the highest HOTA, ByteTrack is faster and yields higher HOTA on SOMPT22 and MOT20, while BoT-SORT reduces IDSW and Frag. Nevertheless, the leap from correlation to mechanism is weak. The authors hypothesize that RT-DETR’s global attention helps dense scenes but acknowledge it is unverified. Prior work is cited fairly for context, though there is no direct reconciliation with earlier fine-tuned detector-tracker surveys. The cited RT-DETR architecture description matches the original source.
The authors publish code and report key settings—640 px input, confidence threshold 0.25, NMS IoU 0.7, and BoxMOT default tracker parameters—which aids reproduction. Yet several details are missing: only three sequences per benchmark are used instead of the full official sets; random seeds are not reported; PyTorch or TensorRT backend specifics are omitted; and hardware-specific FPS on an RTX 4070 Ti Super are not transferable. The exact Ultralytics API invocation that distinguishes YOLO26 NMS from end-to-end mode is also not shown.
Pedestrian tracking in video is important for applications such as surveillance and scene analysis, but it becomes more challenging in crowded scenarios with frequent occlusions and rapid motion. This paper analyzes how detector architecture, model scale, scene density, and inference regime affect the tracking of multiple pedestrians. We compare the YOLO11, RT-DETR, and YOLO26 detector families with the ByteTrack and BoT-SORT trackers on sequences from the MOT17, MOT20, and SOMPT22 benchmarks. For YOLO26, we also compare traditional overlap filtering via Non-Maximum Suppression (NMS) with end-to-end inference. The RT-DETR family achieved the highest mean scores on the HOTA tracking quality metric across all three benchmarks with both trackers, with a more pronounced advantage on MOT20, albeit at a higher computational cost. On MOT17 and SOMPT22, YOLO26 variants, especially YOLO26-L with NMS, offered favorable combinations of quality and speed compared with RT-DETR. In the YOLO26 inference comparison, the end-to-end mode did not yield consistent tracking gains over the NMS-based regime. Taken together, these results provide a practical guide for selecting detectors and trackers for this task based on application priorities.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.