摘要
Audio-Visual Speech Recognition (AVSR) achieves strong performance in controlled conditions, but its accuracy drops sharply when audio or video streams are degraded by noise, blur, or occlusion. Existing fusion strategies often rely on fixed weighting or attention mechanisms, which cannot reliably adapt to per-sample modality quality and are easily biased toward corrupted inputs. To address this limitation, we propose DC-AVSR, a dynamic collaborative framework for robust multimodal speech recognition. The framework comprises two key modules: a Dynamic Contribution Learning Network (DCLN) that leverages prediction loss as a weak supervisory signal for sample-level reliability estimation, and a Collaborative Compensatory Update (CCU) mechanism that encourages learners to correct each other's errors through error-driven sample reweighting. Experiments on LRS2 and LRS3 show that DC-AVSR maintains competitive accuracy under clean inputs and achieves significant improvements under diverse distortion scenarios. These results demonstrate the effectiveness of loss-driven adaptive fusion and collaborative learning for building reliable audio-visual speech recognition systems in real-world environments.
| 源语言 | 英语 |
|---|---|
| 期刊论文编号 | 134694 |
| 期刊 | Neurocomputing |
| 卷 | 702 |
| DOI | |
| 出版状态 | 已出版 - 14 11月 2026 |
学术指纹
探究 'DC-AVSR: A dynamic collaborative approach for robust audio-visual speech recognition under multimodal distortions' 的科研主题。它们共同构成独一无二的学术指纹。引用此
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver