Skip to main navigation Skip to search Skip to main content

DBMC: Dynamic Balance Modality Control in Deep Reinforcement Learning

  • Northwestern Polytechnical University Xian

Research output: Contribution to journal › Article › peer-review

Abstract

Multimodal reinforcement learning has gained attention due to its ability to integrate information from different modalities, enhancing the agent's perception of the environment. However, differences in data distribution, informational content, and significance across modalities lead to imbalances in the training process of the modality encoders. To address this, we propose a Dynamic Balance Modality Control (DBMC) approach, which uses the dormant ratio of modality encoders as a metric to quantify the training progress of each modality encoder and adjusts the gradients to regulate the performance of different modalities. Experiments conducted in the DeepMind Control Suite (DMC) and Robosuite environments show a clear association between the dormant ratio and encoder performance. In the Stand task, when the encoder's dormant ratio is fixed and does not update during training, the encoder with a lower dormant ratio achieves a final cumulative reward that is 44.7% higher than that of the one with a higher dormant ratio. Meanwhile, in the Swingup task, the encoder with a high dormant ratio fails to achieve reward convergence, indicating severe performance degradation. When integrated with DrQ, DBMC improves cumulative rewards by 21.9% and 10.9% on the Stand and Swingup tasks in DMC, respectively, and by 50.0%, 9.6%, and 83.8% on the Lift, Wipe, and Door tasks in Robosuite, respectively. Furthermore, DBMC can be readily integrated into DrQ, SAC-AE, CURL, and DrQv2, and remains effective in tasks involving more modalities.

Original languageEnglish
Pages (from-to)12807-12814
Number of pages8
JournalIEEE Robotics and Automation Letters
Volume11
Issue number11
DOIs
StatePublished - 1 Nov 2026

Keywords

  • Reinforcement learning
  • machine learning for robot control
  • sensor fusion

Fingerprint

Dive into the research topics of 'DBMC: Dynamic Balance Modality Control in Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this