Publications

<title>Abstract</title> Visual feedback derived from digital subtraction angiography (DSA) images is indispensable for guiding interventional decisions during vascular procedures, as real-time DSA imaging provides critical information about the precise positioning of instruments within the vasculature. Excessive bending of the guidewire or microwire tip can pose significant risks, including potential damage to the intimal surface. To address this challenge, this study aims to integrate DSA-based visual feedback into the closed-loop control of an endovascular robotic system, thereby enhancing its safety and intelligence to prevent excessive wire bending. Building on prior work, we propose a distributed hardware and software architecture for the robotic system. This architecture enables robust support for various combinations of interventional instruments while ensuring efficient and reliable transmission of data and commands between system modules. A key innovation is the introduction of vision-based collaborative assistance, which utilizes DSA images to segment the interventional instrument and calculate its bending energy based on the wire’s curvature. This calculated energy is then constrained through nonlinear thresholds to establish a closed-loop control logic. The vascular interventional robotic system was deployed in a standard interventional operating room and evaluated through experiments involving vascular phantoms and animal models. These evaluations demonstrated the system’s capabilities in teleoperation, low communication latency across modules, and improved procedural safety and intelligence facilitated by the vision co-pilot. Quantitative results indicated that the vision co-pilot significantly reduced excessive bending of the microwire tip, enabling smoother, safer, and more efficient completion of interventional tasks.

Effective object handovers from humans-to-robots (H2R) in multi-scenario environments are a key challenge in human-robot collaboration. Numerous studies have explored various perception strategies. However, robust handover in complex settings remains challenging due to frequent lighting interference to RGB-based semantic segmentation, or inherent blindspot issue for limited perceptual views. Under these difficulties, this paper proposes a new perception framework for human-robot handover under eye-on-scene setting. Firstly, this framework uses a single-pixel RGB tracking for initial localization, a point cloud segmentation with regulated extraction, and a learning-based completion for camera blind spots. It works reliably encountering RGB overexposure and shows clear improvement in human handover postures at blindspot. Then, the method introduces a KNN-based loss to improve RandLA-Net segmentation, and an object feature classification strategy to enhance ODGNet-based completion. These boost recognition efficiency and generalization in real handover tasks. Finally, in handover experiments on 15 novel objects, our system achieved an overall success rate of 88.5%. In the quantitative evaluation involving four objects and five representative human handover postures, our method demonstrates improvements over the No Completion baseline in terms of both the overall success rate and the number of predicted valid grasp poses. When encountering RGB overexposure, our method exhibits a more significant advantage in perceptual robustness and grasping effectiveness compared to traditional RGB-based segmentation methods.

BackgroundFor the interactions in virtual reality, it is essential to map the user’s physical motion to the avatar in the virtual world. While reliable lower-limb motions are available under pre-installed cameras, the range of the walking motion is limited by the infrastructures.MethodWe propose a new wearable solution to reproduce the lower-limb motions and map them to the virtual avatar in real time. We employ a single depth camera and design a waist-wearable layout to capture the lower-limb motions relative to the waist. By exploiting the vision data observed by the camera, we further estimate the global velocity of the user.ResultsExperiments are carried out to verify our solution. We quantitatively evaluate the estimated global velocity with an optical motion capture system. We also map the recovered lower-limb motion to the avatar and utilize a standard questionnaire to measure the sense of embodiment. The experiments show that our wearable solution are feasible and effective, being applicable to different people from the perceptual perspective.ConclusionsThe results verify that users are allowed to naturally explore the virtual world with the embodiment using the lightweight equipment.

This letter presents a snake robot with a gripper for inspection and maintenance in narrow spaces. The proposed robot has a gripper equipped with a camera and a laser distance sensor that can inspect the surroundings and grasp objects. The control methods of the proposed robot consist of three parts: 1) A double-layer controller based on central pattern generator (CPG) realizes the locomotion of the robot in narrow spaces. 2) A circular posture is proposed to avoid rollover and improve the payload capacity when the robot is grasping or manipulating an object on either side. 3) A three-stage grasping strategy is proposed to realize semi-automatic grasping operations. A prototype of the robot has been developed and the effectiveness of the proposed control methods have been verified in the payload capacity test, target grasping test, and pick-transport-place experiment.

Tendon-sheath system (TSS) provides a simple yet dexterous solution of force transmission for remote actuation. However, the nonlinear friction determines the control performance of TSS. This paper proposes a simplified piecewise linear model to construct backlash hysteresis. An inverse transmission of this model is used to control distal-end force. Considering the effect of velocity on the transmission model, a piecewise inverse mode method with variable parameters is designed to reduce the force tracking error. The performance of the proposed methods is evaluated by force-tracking experiments with different velocities. The results show that the proposed compensator achieves good force-tracking performance and reduces root mean square error (RMSE) from 3.95 N (without compensator), 1.03 N (traditional compensator), and 0.79 N (piecewise compensator) to 0.43 N (piecewise compensator with varying parameters).