Exploiting temporal consistency for real-time video depth estimation

Haokui Zhang, Chunhua Shen, Ying Li, Yuanzhouhan Cao, Yu Liu, Youliang Yan

科研成果: 书/报告/会议事项章节会议稿件同行评审

94 引用 (Scopus)

摘要

Accuracy of depth estimation from static images has been significantly improved recently, by exploiting hierarchical features from deep convolutional neural networks (CNNs). Compared with static images, vast information exists among video frames and can be exploited to improve the depth estimation performance. In this work, we focus on exploring temporal information from monocular videos for depth estimation. Specifically, we take the advantage of convolutional long short-term memory (CLSTM) and propose a novel spatial-temporal CSLTM (ST-CLSTM) structure. Our ST-CLSTM structure can capture not only the spatial features but also the temporal correlations/consistency among consecutive video frames with negligible increase in computational cost. Additionally, in order to maintain the temporal consistency among the estimated depth frames, we apply the generative adversarial learning scheme and design a temporal consistency loss. The temporal consistency loss is combined with the spatial loss to update the model in an end-to-end fashion. By taking advantage of the temporal information, we build a video depth estimation framework that runs in real-time and generates visually pleasant results. Moreover, our approach is flexible and can be generalized to most existing depth estimation frameworks. Code is available at: Https://tinyurl.com/STCLSTM.

源语言英语
主期刊名Proceedings - 2019 International Conference on Computer Vision, ICCV 2019
出版商Institute of Electrical and Electronics Engineers Inc.
1725-1734
页数10
ISBN(电子版)9781728148038
DOI
出版状态已出版 - 10月 2019
活动17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, 韩国
期限: 27 10月 20192 11月 2019

出版系列

姓名Proceedings of the IEEE International Conference on Computer Vision
ISSN(印刷版)1550-5499

会议

会议17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
国家/地区韩国
Seoul
时期27/10/192/11/19

指纹

探究 'Exploiting temporal consistency for real-time video depth estimation' 的科研主题。它们共同构成独一无二的指纹。

引用此