Non-destructive Ripeness Detection of Avocados (Persea Americana Mill) using Vision and Tactile Perception Information Fusion Method

Vision (skin color) and tactile (firmness) characteristics of avocado are important characteristics associated with the level of ripeness. Avocados do not soften uniformly during ripening, and it is difficult to measure the firmness value at each location. Machine learning-based visual characteristic grading is difficult to analyze quantitatively. It works poorly for more refined grading and is better suited for coarse grading. In addition, there are asynchronous changes in the tactile and vision characteristics of avocado fruit during the ripening period. In this study, combining the tactile-based ripeness grading technique with the vision-based ripeness grading technique is proposed to obtain more stable and reliable grading results. In the first phase, visual characteristic (skin color) of avocado images is graded based on the ResNet-34 model, and three maturity classes (A, B and C) were initially identified. The second stage uses a pneumatic flexible sensing soft manipulator. It integrates four flexible pressure sensors to grasp avocados one by one and sense their firmness. The second stage is subdivided into six maturity classes (A1, A2, B1, B2, C1, C2) based on the first stage. This study achieves more refined grading (6 levels) and high accuracy (96.0% grading success rate), which is superior to visual or tactile grading only and manual maturity grading commonly used in current production.