Grapevine winter pruning: Merging 2D segmentation and 3D point clouds for pruning point generation

Grapevine winter pruning is a labor-intensive and repetitive process that significantly influences grape yield and quality at harvest and produced wine. Due to its complexity and repetitive nature, the task demands skilled labor that needs to be trained, as in many other agricultural sectors. This paper encompasses an approach that targets using a robotic system to perform autonomous grapevine winter pruning using a vision system and artificial intelligence. In our previous work, we presented a 2D neural network that segmented images of grapevines into 5 different classes of plant organs during their dormant season. In this paper, we expand into the third dimension, introducing point clouds into our algorithm. The 3D approach creates instance-segmented point clouds using depth images and segmentation masks obtained with our 2D neural network. After the 3D reconstruction, the system extracts thickness measurement and uses agronomic knowledge to place pruning points for balanced pruning. The study not only delineates the integration of 2D and 3D methods but also scrutinizes their efficacy in pruning point identification. The real-world performance of the created system was evaluated and statistically analyzed on data collected during field trials in the winter pruning season 2022/2023, where the system was used in a potted vineyard to prune a set of test vines, where the positive success rate is 54.2%. Moreover, as one of the main contributions, the paper underscores a unique facet of adaptability, presenting a customizable framework that empowers end-users to fine-tune parameters according to the expected balanced pruning. This adaptability extends to variables such as the number of nodes to retain on pruned spurs and the preferred cane thickness, encapsulating the versatility of the 3D approach.