An Improved SAC-Based Deep Reinforcement Learning Framework for Collaborative Pushing and Grasping in Underwater Environments

Jian Gao, Yufeng Li, Yimin Chen, Yaozhen He, Jingwei Guo

Research output: Contribution to journalArticlepeer-review

11 Scopus citations

Abstract

Autonomous grasping is a fundamental task for underwater robots, but direct grasping for tightly stacked objects will lead to collisions and grasp failures, which requires pushing actions to separate the target object and increase grasp success (GS) rates. Hence, this article proposes a novel approach by employing an improved soft actor-critic (SAC) algorithm within a deep reinforcement learning (RL) framework for achieving collaborative pushing and grasping actions. The developed scheme employs an end-to-end control strategy that maps input images to actions. Specifically, an attention mechanism is introduced in the visual perception module to extract the necessary features for pushing and grasping actions to enhance the training strategy. Moreover, a novel pushing reward function is designed, comprising a per-object distribution function around the target and a global object distribution assessment network named PA-Net. Furthermore, an enhanced experience replay strategy is introduced to address the sparsity issue of grasp action rewards. Finally, a training environment for underwater manipulators is established, in which variations in light, water flow noise, and pressure effects are incorporated to simulate underwater work conditions more realistically. The simulation and real-world experiments demonstrate that the proposed learning strategy efficiently separates target objects and avoids inefficient pushing actions, achieving a significantly higher GS rate.

Original languageEnglish
Article number2512814
Pages (from-to)1-14
Number of pages14
JournalIEEE Transactions on Instrumentation and Measurement
Volume73
DOIs
StatePublished - 2024

Keywords

  • Attention mechanism
  • collaborative actions
  • deep reinforcement learning (RL)
  • pushing-grasping
  • reward function
  • underwater manipulator

Fingerprint

Dive into the research topics of 'An Improved SAC-Based Deep Reinforcement Learning Framework for Collaborative Pushing and Grasping in Underwater Environments'. Together they form a unique fingerprint.

Cite this