Point Cloud Perception in Human-to-Robot Handover based on Semantic Segmentation and Shape Completion

Effective object handovers from humans-to-robots (H2R) in multi-scenario environments are a key challenge in human-robot collaboration. Numerous studies have explored various perception strategies. However, robust handover in complex settings remains challenging due to frequent lighting interference to RGB-based semantic segmentation, or inherent blindspot issue for limited perceptual views. Under these difficulties, this paper proposes a new perception framework for human-robot handover under eye-on-scene setting. Firstly, this framework uses a single-pixel RGB tracking for initial localization, a point cloud segmentation with regulated extraction, and a learning-based completion for camera blind spots. It works reliably encountering RGB overexposure and shows clear improvement in human handover postures at blindspot. Then, the method introduces a KNN-based loss to improve RandLA-Net segmentation, and an object feature classification strategy to enhance ODGNet-based completion. These boost recognition efficiency and generalization in real handover tasks. Finally, in handover experiments on 15 novel objects, our system achieved an overall success rate of 88.5%. In the quantitative evaluation involving four objects and five representative human handover postures, our method demonstrates improvements over the No Completion baseline in terms of both the overall success rate and the number of predicted valid grasp poses. When encountering RGB overexposure, our method exhibits a more significant advantage in perceptual robustness and grasping effectiveness compared to traditional RGB-based segmentation methods.