Context-Adaptive Online Reinforcement Learning for Multi-view Video Summarization on Mobile Devices

The huge amount of video data produced by ubiqui tous cameras imposes significant challenges for users to efficiently obtain useful video information. Multi-view video summarization (MVS) aggregates multi-view videos into information-rich video summaries by considering content correlations within each view and between multiple views. Existing MVS methods fail to concentrate on performance across scenarios and usually achieve satisfactory performance on specific training datasets. However, when faced with unseen video scenarios, the quality of the summaries generated by existing methods may degrade. Moreover, they usually only use cameras for data acquisition, which require a large amount of network bandwidth to transfer the data to the server for processing. To bridge this gap, we propose a context-adaptive online reinforcement learning multi-view video summarization framework (COORS) that meets the low response latency performance requirements of context adaptation while ensuring camera hardware compatibility. Specifically, COORS enables retraining in new contexts by extracting contextindependent rewards, while improving model convergence speed based on representation learning and replica playback. Extensive experiments show that COORS has better performance compared to the state-of-the-art baselines.