Deep learning-based pose prediction for visual servoing of robotic manipulators using image similarity

Yaozhen He, Jian Gao, Yimin Chen

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

The accuracy of pose prediction is crucial in learning-based visual servoing. Motivated by the fact that the more similar observed images are, the closer the camera poses, we propose a joint training strategy with a two-part loss function in this paper. One part is the least absolute deviation (L1) loss function, which is defined by the error between the predicted pose and the pose label. The other is the mean similarity image measurement loss function (MSIM), which is related to the image's brightness, contrast, and structure similarity and is determined by the differences between the input image and the image corresponding to the predicted pose. Meanwhile, a data generator based on spherical projection is created to generate data uniformly for training a CNN model, and position-based visual servoing (PBVS) is designed for a robotic manipulator after pose prediction. A numeric simulation and real experiments are conducted in a virtual environment and with a UR3 manipulator. The results show that the proposed method can realize more accurate pose prediction and is robust to occlusion disturbance, and PBVS is achieved by using monocular images.

Original languageEnglish
Pages (from-to)343-352
Number of pages10
JournalNeurocomputing
Volume491
DOIs
StatePublished - 28 Jun 2022

Keywords

  • PBVS
  • Pose prediction
  • Similarity measurement
  • Visual servoing

Fingerprint

Dive into the research topics of 'Deep learning-based pose prediction for visual servoing of robotic manipulators using image similarity'. Together they form a unique fingerprint.

Cite this