Robust Multi-Object Tracking Using Vision Sensor with Fine-Grained Cues in Occluded and Dynamic Scenes

Multi-object tracking (MOT) using vision sensors remains a challenging problem, particularly in dynamic backgrounds and severe occlusions. Existing methods, relying on holistic appearance or spatial cues, fail to capture detailed information within object regions and the background, resulting in inaccurate and inconsistent tracking. To address these issues, we propose a novel Detail-Driven Multi-Object Tracking (DD-MOT) method that revisits and leverages fine-grained cues to discover the dynamics of both object regions and the background, facilitating the recovery and association of trajectories. Specifically, the proposed method consists of three key modules: (i) Points Trajectories Generator, (ii) Camera Motion and Occlusion Compensation, and (iii) Fine- and Coarsegrained Association. The Points Trajectories Generator is responsible for generating fine-grained cues by sampling an initial set of points, generating point trajectories, and refining the initial set of points. The Camera Motion and Occlusion Compensation module utilizes the background and object point trajectories to correct background motion and recover occluded object bounding boxes. Finally, the Fine- and Coarse-grained Association module leverages point trajectory cues to assist in establishing a more effective association strategy, combining both fine-grained and coarse-grained spatial cues from the object bounding boxes. We evaluate our DD-MOT method on several benchmark datasets, including MOT17, MOT20, demonstrating that it consistently outperforms state-of-the-art (SOTA) methods in key metrics such as HOTA (64.2, 62.6), MOTA (80.8, 78.1), and IDF1 (78.8, 77.1).