Point cloud augmented virtual reality environment with haptic constraints for teleoperation

Remote manipulation of a robot without assistance in an unstructured environment is a challenging task for operators. In this paper, a novel methodology for haptic constraints in a point cloud augmented virtual reality environment is proposed to address this human operation limitation. The proposed method generates haptic constraints in real time for an unstructured environment, including regional constraints and guidance constraints. A modified implicit surface method is applied for regional constraint generation for the entire point cloud. Additionally, the isosurface derived from the implicit surface is proposed for real-time three-dimensional artificial force field estimation. For guidance constraint generation, a new incremental prediction and local artificial force field generation method based on the modified sigmoid model is proposed in an unstructured point cloud virtual reality environment. With the generated haptic constraints, the operator can control the robot to realize obstacle avoidance and easily reach the target tasks. System evaluation is conducted, and the result demonstrates the effectiveness of the proposed method. In addition, a 10-participant study with users who control the robot to three specific targets shows that the system can enhance human operation efficiency and reduce time costs by at least 59% compared with no-haptic-constraint operations. Additionally, the designed questionnaire also demonstrates that the proposed methodology can reduce the workload during human operations.