TY - JOUR
T1 - 3DGAM
T2 - using 3D gesture and CAD models for training on mixed reality remote collaboration
AU - Wang, Peng
AU - Bai, Xiaoliang
AU - Billinghurst, Mark
AU - Zhang, Shusheng
AU - Wei, Sili
AU - Xu, Guangyao
AU - He, Weiping
AU - Zhang, Xiangyu
AU - Zhang, Jie
N1 - Publisher Copyright:
© 2020, Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2021/8
Y1 - 2021/8
N2 - As Virtual Reality(VR), Augmented Reality(AR), Mixed Reality(MR) technology becomes more accessible, it is important to explore VR/AR/MR technologies that can be used for remote collaboration on physical tasks. Previous research has shown that gesture-based interaction is intuitive and expressive for remote collaboration, and using 3D CAD models can provide clear instructions for assembly tasks. In this paper, therefore, we describe a new MR remote collaboration system which combines the use of gesture and CAD models in a complementary manner. The prototype system enables a remote expert in VR to provide instructions based on 3D gesture and CAD models (3DGAM) for a local worker who uses AR to see these instructions. Using this interface, we conducted a formal user study to explore the effect of sharing 3D gesture and CAD models in an assembly training task. We found that the combination of 3D gesture and CAD models can improve remote collaboration on an assembly task with respect to the performance time and user experience. Finally, we provide some conclusions and directions for future research.
AB - As Virtual Reality(VR), Augmented Reality(AR), Mixed Reality(MR) technology becomes more accessible, it is important to explore VR/AR/MR technologies that can be used for remote collaboration on physical tasks. Previous research has shown that gesture-based interaction is intuitive and expressive for remote collaboration, and using 3D CAD models can provide clear instructions for assembly tasks. In this paper, therefore, we describe a new MR remote collaboration system which combines the use of gesture and CAD models in a complementary manner. The prototype system enables a remote expert in VR to provide instructions based on 3D gesture and CAD models (3DGAM) for a local worker who uses AR to see these instructions. Using this interface, we conducted a formal user study to explore the effect of sharing 3D gesture and CAD models in an assembly training task. We found that the combination of 3D gesture and CAD models can improve remote collaboration on an assembly task with respect to the performance time and user experience. Finally, we provide some conclusions and directions for future research.
KW - 3D CAD models
KW - Augmented reality
KW - Mixed reality
KW - Physical tasks
KW - Remote collaboration
KW - Sharing gesture
UR - http://www.scopus.com/inward/record.url?scp=85090244361&partnerID=8YFLogxK
U2 - 10.1007/s11042-020-09731-7
DO - 10.1007/s11042-020-09731-7
M3 - 文章
AN - SCOPUS:85090244361
SN - 1380-7501
VL - 80
SP - 31059
EP - 31084
JO - Multimedia Tools and Applications
JF - Multimedia Tools and Applications
IS - 20
ER -