跳到主要导航 跳到搜索 跳到主要内容

DBMC: Dynamic Balance Modality Control in Deep Reinforcement Learning

  • Northwestern Polytechnical University Xian

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Multimodal reinforcement learning has gained attention due to its ability to integrate information from different modalities, enhancing the agent's perception of the environment. However, differences in data distribution, informational content, and significance across modalities lead to imbalances in the training process of the modality encoders. To address this, we propose a Dynamic Balance Modality Control (DBMC) approach, which uses the dormant ratio of modality encoders as a metric to quantify the training progress of each modality encoder and adjusts the gradients to regulate the performance of different modalities. Experiments conducted in the DeepMind Control Suite (DMC) and Robosuite environments show a clear association between the dormant ratio and encoder performance. In the Stand task, when the encoder's dormant ratio is fixed and does not update during training, the encoder with a lower dormant ratio achieves a final cumulative reward that is 44.7% higher than that of the one with a higher dormant ratio. Meanwhile, in the Swingup task, the encoder with a high dormant ratio fails to achieve reward convergence, indicating severe performance degradation. When integrated with DrQ, DBMC improves cumulative rewards by 21.9% and 10.9% on the Stand and Swingup tasks in DMC, respectively, and by 50.0%, 9.6%, and 83.8% on the Lift, Wipe, and Door tasks in Robosuite, respectively. Furthermore, DBMC can be readily integrated into DrQ, SAC-AE, CURL, and DrQv2, and remains effective in tasks involving more modalities.

源语言英语
页(从-至)12807-12814
页数8
期刊IEEE Robotics and Automation Letters
卷11
期11
DOI
出版状态已出版 - 1 11月 2026

学术指纹

探究 'DBMC: Dynamic Balance Modality Control in Deep Reinforcement Learning' 的科研主题。它们共同构成独一无二的学术指纹。

引用此