Automatic 3-D Imaging and Measurement of Human Spines with a Robotic Ultrasound System

Comparing with X-ray, ultrasound imaging has the advantages of no radiation and easy operation, etc., which is expected to be an effective alternative in clinical assessment for the human spine. At present, freehand scanning is mainly used in ultrasound imaging field, and recently robot-assisted scanning has emerged as a promising way to gradually reduce human intervention. However, the automatic recognition and scanning, as well as the control strategy of robots that guarantee the tight coupling between a probe and human skin, are still challenges. Moreover, the current measurement method of the spine is usually conducted based on plain radiographs, which lacks spatial structure information of spine. In this article, a new radiation-free robotic ultrasound system for 3-D imaging and a measurement method of the spine based on 3-D imaging are presented. A fully convolutional network (FCN) named fuse-Unet based on RGB and depth images captured by an RGB-depth (RGB-D) sensor Kinect, is proposed to realize the automatic recognition of human spine area and scanning path pre-planning. Then a six-degree-of-freedom robotic arm plays the role of a doctor to complete the automatic scanning along the pre-planning path, during which a normal-vector-based method and two force sensors are used to ensure that the probe well fits the spine area. Finally, 3-D ultrasound reconstruction and visualization of the spine are realized, based on which cobb angles are calculated for assessing the morphological structure of the spine. Phantom and in vivo experiments are conducted for evaluating the performance of the proposed system. Experimental results validate the practicability and accuracy of the system, indicating its potential for clinical application.