Context-Adaptive Online Reinforcement Learning for Multi-view Video Summarization on Mobile Devices

Jingyi Hao, Sicong Liu, Bin Guo, Yasan Ding, Zhiwen Yu

科研成果: 书/报告/会议事项章节会议稿件同行评审

摘要

The huge amount of video data produced by ubiqui tous cameras imposes significant challenges for users to efficiently obtain useful video information. Multi-view video summarization (MVS) aggregates multi-view videos into information-rich video summaries by considering content correlations within each view and between multiple views. Existing MVS methods fail to concentrate on performance across scenarios and usually achieve satisfactory performance on specific training datasets. However, when faced with unseen video scenarios, the quality of the summaries generated by existing methods may degrade. Moreover, they usually only use cameras for data acquisition, which require a large amount of network bandwidth to transfer the data to the server for processing. To bridge this gap, we propose a context-adaptive online reinforcement learning multi-view video summarization framework (COORS) that meets the low response latency performance requirements of context adaptation while ensuring camera hardware compatibility. Specifically, COORS enables retraining in new contexts by extracting contextindependent rewards, while improving model convergence speed based on representation learning and replica playback. Extensive experiments show that COORS has better performance compared to the state-of-the-art baselines.

源语言英语
主期刊名Proceedings - 2022 IEEE 28th International Conference on Parallel and Distributed Systems, ICPADS 2022
出版商IEEE Computer Society
411-418
页数8
ISBN(电子版)9781665473156
DOI
出版状态已出版 - 2023
活动28th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2022 - Nanjing, 中国
期限: 10 1月 202312 1月 2023

出版系列

姓名Proceedings of the International Conference on Parallel and Distributed Systems - ICPADS
2023-January
ISSN(印刷版)1521-9097

会议

会议28th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2022
国家/地区中国
Nanjing
时期10/01/2312/01/23

指纹

探究 'Context-Adaptive Online Reinforcement Learning for Multi-view Video Summarization on Mobile Devices' 的科研主题。它们共同构成独一无二的指纹。

引用此