DecoderTracker: Decoder-only end-to-end method for multiple-object tracking

Decoder-only Transformer architectures, such as GPT, have demonstrated superior performance in many areas compared to traditional encoder-decoder structure transformer methods. Over the years, end-to-end methods based on the traditional transformer structure, like MOTR, have achieved remarkable performance in multi-object tracking. However, these methods suffer from substantial computational costs and optimization challenges inherent to dynamic data processing, leading to suboptimal inference speeds and prolonged training times. To address the aforementioned issues, this paper optimized the network architecture and proposed an effective training strategy to mitigate the problem of prolonged training times, thereby developing DecoderTracker, a novel end-to-end tracking method. Subsequently, to tackle the optimization challenges arising from dynamic data, this paper introduced FixDT by incorporating a Fixed-Size Query Memory and refining certain attention layers. Our methods, outperforms MOTR on multiple benchmarks without incorporating complex heuristic components, featuring a 2 to 3 times faster inference than MOTR, respectively. The proposed method is implemented in open-source code, accessible at https://github.com/liaopan-lp/MO-YOLO.