Comparison of visual and multisensory augmented reality for precise manual manipulation tasks

Precise manual manipulation is an important skill in daily life, and Augmented Reality (AR) is increasingly being used to support such operations. This article reports on a study investigating the usability of visual and multisensory AR for precise manual manipulation tasks, in particular the representation of detailed deviations from the target pose. Two AR instruction interfaces were developed: the visual deviation instruction and the multisensory deviation instruction. Both interfaces used visual cues to indicate the required directions for manipulation. The difference was that the visual deviation instruction used text and color mapping to represent deviations, whereas the multisensory deviation instruction used sonification and vibration to represent deviations. A user study was conducted with 16 participants to compare the two interfaces. The results found a significant difference only in speed, without significant differences in accuracy, perceived ease-of-use, workload, or custom user experience elements. Multisensory deviation cues can speed up precise manual manipulation compared to visual deviation cues, but inappropriate sonification and vibration strategies can negatively affect users’ subjective experience, offsetting the benefits of multisensory AR. Based on the results, several recommendations were provided for designing AR instruction interfaces to support precise manual manipulation.