跳到主要导航 跳到搜索 跳到主要内容

Where Does It Exist from the Low-Altitude: Spatial Aerial Video Grounding

  • Northwestern Polytechnical University Xian

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

4 引用 (Scopus)

摘要

The task of localizing an object's spatial tube based on language instructions and video, known as spatial video grounding (SVG), has attracted widespread interest. Existing SVG tasks have focused on ego-centric fixed front perspective and simple scenes, which only involved a very limited view and environment. However, UAV-based SVG remains underexplored, which neglects the inherent disparities in drone movement and the complexity of aerial object localization. To facilitate research in this field, we introduce the novel spatial aerial video grounding (SAVG) task. Specifically, we meticulously construct a large-scale benchmark, UAV-SVG, which contains over 2 million frames and offers 216 highly diverse target categories. To address the disparities and challenges posed by complex aerial environments, we propose a new end-to-end transformer architecture, coined SAVG-DETR. The innovations are three-fold. 1) To overcome the computational explosion of self-attention when introducing multi-scale features, our encoder efficiently decouples the multi-modality and multi-scale spatio-temporal modeling into intra-scale multi-modality interaction and cross-scale visual-only fusion. 2) To enhance small object grounding ability, we propose the language modulation module to integrate multi-scale information into language features and the multilevel progressive spatial decoder to decode from high to low level. The decoding stage for the lower-level vision-language features is gradually increased. 3) To improve the prediction consistency across frames, we design the decoding paradigm based on offset generation. At each decoding stage, we utilize reference anchors to constrict the grounding region, use context-rich object queries to predict offsets, and update reference anchors for the next stage. From coarse to fine, our SAVG-DETR gradually bridges the modality gap and iteratively refines reference anchors of the referred object, eventually grounding the spatial tube. Extensive experiments demonstrate that our SAVG-DETR significantly outperforms existing state-of-the-art methods. The dataset and code will be available at here.

源语言英语
主期刊名39th Conference on Neural Information Processing Systems, NeurIPS 2025
出版商Neural information processing systems foundation
页92030-92064
页数35
ISBN(电子版)9798331338275
出版状态已出版 - 2025
活动39th Conference on Neural Information Processing Systems, NeurIPS 2025 - San Diego, 美国
期限: 2 12月 2025 → 7 12月 2025

丛书

姓名Advances in Neural Information Processing Systems
卷38
ISSN(印刷版)1049-5258

会议

会议39th Conference on Neural Information Processing Systems, NeurIPS 2025
国家/地区美国
市San Diego
时期2/12/25 → 7/12/25

学术指纹

探究 'Where Does It Exist from the Low-Altitude: Spatial Aerial Video Grounding' 的科研主题。它们共同构成独一无二的学术指纹。

引用此