Projected augmented reality assembly assistance system supporting multi-modal interaction

Jie Zhang, Shuxia Wang, Weiping He, Jianghong Li, Zhiwei Cao, Bingzhao Wei

Research output: Contribution to journalArticlepeer-review

14 Scopus citations

Abstract

As an application paradigm of augmented reality in industry, the projected augmented reality (PAR) system is helpful in assembly, improving the ability of human–computer assembly cooperation. Precious research mainly focuses on visualization whereas the problem of how users naturally interact with the system has still not been discussed. In this paper, we developed a PAR system supporting multi-modal interaction (MMI), with which users could assemble and interact naturally. First, we tested the performance of MMI (speech, gesture, and finger click) against the baseline condition of single mode interaction. Meanwhile, we also compare the performance difference between gesture and finger click in bare-hand interaction. It is found that finger click shows a higher success rate, faster response time, lower false errors, less instruction memory, and better user experience than the gesture in bare-hand interaction. In specific, when the assembly must be completed but the hands are occupied, MMI can take advantage of user interaction modalities and reduce the interaction burden. The results prove that MMI shows better interactive naturalness and performance, which is more suitable for popularization and application in PAR assembly assistance systems. In conclusion, the developed PAR system can be a valid solution for intelligent manufacturing applications.

Original languageEnglish
Pages (from-to)1353-1367
Number of pages15
JournalInternational Journal of Advanced Manufacturing Technology
Volume123
Issue number3-4
DOIs
StatePublished - Nov 2022

Keywords

  • Human-cyber-physical system
  • Industrial applications
  • Multi-modal interaction
  • Natural human computer interaction
  • Projected augmented reality
  • Projection interaction

Fingerprint

Dive into the research topics of 'Projected augmented reality assembly assistance system supporting multi-modal interaction'. Together they form a unique fingerprint.

Cite this