Publications

Layered rhombus-chain-connected model for real-time haptic rendering

Development of a Novel Home Based Multi-Scene Upper Limb Rehabilitation Training and Evaluation System for Post-Stroke Patients

Upper limb rehabilitation requires long-term, repetitive rehabilitation training and assessment. However, many patients cannot afford for heavy medical fees. It is necessary to design an effective, low-cost, and reasonable home rehabilitation and evaluation system. In this paper, we developed a novel home-based multi-scene upper limb rehabilitation training and evaluation system for post-stroke patients. Based on the Kinect sensor and the posture sensor, the multi-sensors fusion method was used to track the motion of the patients. Multiple virtual scenes were designed to encourage rehabilitation training of upper limbs and trunk. A rehabilitation evaluation method was proposed integrating Fugl-Meyer assessment (FMA) scale and upper limb reachable workspace relative surface area (RSA). Furthermore, an FMA-RSA assessment model was established to assess an upper limb motor function. Correlation-based dynamic time warping was used to solve the problem of inconsistent upper limb movement path in different patients. Two clinical trials were conducted. The experimental results show that the system is very friendly to the subjects. The rehabilitation assessment results by this system are highly correlated with the therapist's (the highest forecast accuracy was 92.7% in the 13th item). It also reveals that long-term rehabilitation training can improve the upper limb motor function of the patients statistically significant (p=0.02 <; 0.05). The system has the potential to become an effective home rehabilitation training and evaluation system.

Tracking Error Constraints and Integral Guidance Mechanisms-Based Antidisturbance Path-Tracking Control for a Robotic Fish

Predictor-Based Motion Tracking Control for Cloud Robotic Systems with Delayed Measurements

This paper addresses the problem of motion prediction and tracking control for cloud robotic systems with time-varying delays in measurements. A novel method using an observer-based structure for position and velocity prediction is developed to estimate the real-time information of robot manipulator. The prediction error can converge to zero even if model uncertainties exist in the robot manipulator. Based on the predicted positions and velocities, some sufficient conditions are derived to design suitable tracking controllers such that semi-globally uniformly ultimately bounded tracking performance of the predictor–controller couple can be guaranteed. Finally, the effectiveness and robustness to model uncertainties of the proposed method are verified by a two degree-of-freedom (DOF) robot system.

A rigid and flexible structures combined deployable boom for space exploration

This paper presents a deployable boom which combines a rigid telescopic frame and a flexible tape spring. The front end of the spring is fixed on the rear end of the innermost segment of the frame. The spring spreads and rolls up inside the frame to drive the segments to move one by one to realize the boom deployment and retraction. The driving forces needed to deploy and retract the frame are modeled and simulated. The feasibility of the frame driving by only one spring is also studied. A 1.6 kg prototype system with 2.1 m total deployment length is implemented. Experimental results show the maximum driving forces for deployment and retraction of the frame are about 9.1 N and 6.8 N respectively. The boom is able to deploy in 76 s with energy consumption of 315 J. The boom can resist at least 15 N force axially and 31.5 N·m bending moment when the forces are acted on its front end. Advantages of this kind of boom enable it to be applied for instruments deployment, walking and sampling assists, and robotic arms design in space exploration.

Mean-Based Geodesic Distance Alignment Transfer for Decoding Natural Hand Movement from Mrcps

The Effects of Bilateral Phase-Dependent Closed-Loop Vibration Stimulation With Motor Imagery Paradigm

Vibration stimulation has been shown to have the potential to improve the activation pattern of unilateral motor imagery (MI) and to promote motor recovery. However, in the widely used left and right hand MI brain-computer interface (BCI) paradigm, the vibration stimuli cannot be directly applied to the imaginary side due to the spontaneity of imagery. In this study, we proposed a method of phase-dependent closed-loop vibration stimulation to be applied on both hands, and explored the effects of different vibration stimuli on the left and right hand MI-BCI. Eighteen healthy subjects were recruited and asked to perform, in sequence, MI tasks under three different conditions of vibratory feedback, which were no vibration stimulus (MI), phase-dependent closed-loop vibration stimulus (PDS), and continuous vibration stimulus (CS). Then the performance of the left and right hand MI-BCI and the patterns of brain oscillation were compared and analyzed under these different stimulation conditions. The results showed that vibration stimulation effectively boosted the activation of the sensorimotor cortex and enhanced the functional connectivity among sensorimotor-related brain regions during MI. The closed-loop stimulation evoked stronger event-related desynchronization patterns on the contralateral side of the imagined hand compared to continuous stimulation. There was a more obvious distinction between left hand task and right hand task. In addition, phase-dependent closed-loop vibration stimulation increased classification accuracy by approximately 7% (paired t-test, p=0.004, n=18) compared to MI alone, while continuous vibration stimulation only increased it by 4% (paired t-test, p=0.067, n=18). This result further demonstrated the effectiveness of the phase-dependent closed-loop vibration stimulation method in improving the overall performance of the MI paradigm and is expected to be further applied in areas such as stroke rehabilitation in the future.

Research on the Method of Displaying the Contour Features of Image to the Visually Impaired on the Touch Screen

Conveying image information to the blind or visually impaired (BVI) is an important means to improve their quality of life. The touch screen devices used daily are the potential carriers for BVI to perceive image information through touch. However, touch screen devices also have the disadvantages of limited computing power and lack of rich tactile experience. In order to help BVI to access images conveniently through the touch screen, we built an image contour display system based on vibrotactile feedback. In this paper, an image smoothing algorithm based on convolutional neural network that can run quickly on the touch screen device is first used to preprocess the image to improve the effect of contour extraction. Then, based on the haptic physiological characteristics of human beings, this paper proposes a method of using the improved MH-Pen to guide the BVI to perceive image contour on the touch screen. This paper introduces the extraction and expression methods of image contours in detail, and compares and analyzes the effects of the subjects' perception of image contours in two haptic display modes through two types of user experiments. The experimental results show that the image smoothing algorithm is useful and necessary to help obtain the main contour of the image and to ensure the real-time display of the contour, and the contour expression method based on the motion direction guidance helps the subjects recognize the contour of the image more effectively.

Enhancing Endovascular Interventions with A Robotic System with Vision-Based Collaborative Assistance

<title>Abstract</title> Visual feedback derived from digital subtraction angiography (DSA) images is indispensable for guiding interventional decisions during vascular procedures, as real-time DSA imaging provides critical information about the precise positioning of instruments within the vasculature. Excessive bending of the guidewire or microwire tip can pose significant risks, including potential damage to the intimal surface. To address this challenge, this study aims to integrate DSA-based visual feedback into the closed-loop control of an endovascular robotic system, thereby enhancing its safety and intelligence to prevent excessive wire bending. Building on prior work, we propose a distributed hardware and software architecture for the robotic system. This architecture enables robust support for various combinations of interventional instruments while ensuring efficient and reliable transmission of data and commands between system modules. A key innovation is the introduction of vision-based collaborative assistance, which utilizes DSA images to segment the interventional instrument and calculate its bending energy based on the wire’s curvature. This calculated energy is then constrained through nonlinear thresholds to establish a closed-loop control logic. The vascular interventional robotic system was deployed in a standard interventional operating room and evaluated through experiments involving vascular phantoms and animal models. These evaluations demonstrated the system’s capabilities in teleoperation, low communication latency across modules, and improved procedural safety and intelligence facilitated by the vision co-pilot. Quantitative results indicated that the vision co-pilot significantly reduced excessive bending of the microwire tip, enabling smoother, safer, and more efficient completion of interventional tasks.

Model-Based Load Characteristics Analysis of the Multi-Dimensional Force Sensor

As an important component of force feedback device, multi-dimensional force sensor has been widely used in haptic device, prosthetic hand and other devices. In a given application, the multi-dimensional force sensor is always coupled with a load by the screw thread, which may result in a totally different dynamic behavior from a bare force sensor. It is the key data for deeply understanding the dynamic performance to have information about the structural distribution of stiffness and mass of the multi-dimensional force sensor. To address this need, the approach of model-based load characteristic analysis of the multi-dimensional force sensor is presented in this paper. Dynamic behavior of the force sensor in a given load is described by a lumped mass model consists of spring-mass-damper elements and characterized by the model parameters that describe the dynamic correlation distribution of mass, stiffness, and damping. The main purpose of the proposed approach is the description of the load characteristics of the multi-dimensional force sensor, independent of the given mechanical environment, which provides a certain theoretical reference for the calculation of the load capacity of the force sensor.

Visual–Tactile Fusion Transformer for Grasping and Slip Detection of Unknown Objects

The efficient integration of visual and tactile information is essential for slip detection and evaluation of grasping stability. However, existing research generally combines visual or tactile modalities as prior information, without fully exploring mechanisms for fusing complementary modalities. In this article, we propose a visual–tactile fusion Transformer (VTF-Trans) for slip detection, designed to handle unaligned data in different formats and facilitate cross-modal information exchange. The main advantages of the proposed method are summarized as follows: first, VTF-Trans employs an improved dual-stream transformer for feature extraction. In addition, we introduce a gated modal attention module to further refine cross-modal fusion. Compared with the existing methods, VTF-Trans effectively integrates useful information from different modalities across multiple scales. Second, to extract deep multimodal information, we propose a cross-modal attention (CMA) mechanism. By defining cross-affinity based on single-modality affinities (token metrics), CMA naturally alleviates the gap between different modalities and domains. Third, we evaluate VTF-trans on three datasets and conduct unknown object grasping experiments. Compared with the state-of-the-art methods, VTF-Trans achieves the highest accuracy for robotic grasping and slip detection, highlighting its superior performance and practical applicability.

Texture Feature Extraction Method for Ground Nephogram Based on Hilbert Spectrum of Bidimensional Empirical Mode Decomposition

Abstract It is important to recognize the type of cloud for automatic observation by ground nephoscope. Although cloud shapes are protean, cloud textures are relatively stable and contain rich information. In this paper, a novel method is presented to extract the nephogram feature from the Hilbert spectrum of cloud images using bidimensional empirical mode decomposition (BEMD). Cloud images are first decomposed into several intrinsic mode functions (IMFs) of textural features through BEMD. The IMFs are converted from two- to one-dimensional format, and then the Hilbert–Huang transform is performed to obtain the Hilbert spectrum and the Hilbert marginal spectrum. It is shown that the Hilbert spectrum and the Hilbert marginal spectrum of different types of cloud textural images can be divided into three different frequency bands. A recognition rate of 87.5%–96.97% is achieved through random cloud image testing using this algorithm, indicating the efficiency of the proposed method for cloud nephogram.

Mean-based geodesic distance alignment transfer for decoding natural hand movement from MRCPs

Egocentric-Vision based Hand Posture Control System for Reconnaissance Robots

Uncertainty evaluation study on Chang’E-1 Laser Altimeter on-orbit detection error

The Chang'E-1 Laser Altimeter(LAM), as one of the scientific instruments onboard the Chinese Chang'E-1 orbiter, has successfully gained the massive lunar elevation scientific data of global topography of the moon. Uncertainty evaluation of the lunar elevation detection error based on LAM scientific data is developed in this paper. Firstly, the data are selected from the flat terrain region in all the lunar elevation detection data; Secondly, after the pseudo elevation data are removed in the selected region, regional elevation mean and standard deviation are calculated. Making use of the calculations and taking into account all kinds of uncertainty contributors of LAM orbiting exploring, the uncertainty evaluation methods of the LAM in-orbit elevation exploring are proposed on the basis of the guide to Monte Carlo Methods. Finally, the uncertainty evaluation results of different regions of lunar surface are given. The evaluation results not only can provide the basis for further analysis laser altimeter measurement error sources, but also give the reference for making the high precision moon digital elevation graph and provide theoretical guidance for the accuracy requirement of design of payload on lunar orbiter.

Real-Time Lower-Limb Motion Embodiment in Virtual Reality from a Single Waist-Wearable Camera

BackgroundFor the interactions in virtual reality, it is essential to map the user’s physical motion to the avatar in the virtual world. While reliable lower-limb motions are available under pre-installed cameras, the range of the walking motion is limited by the infrastructures.MethodWe propose a new wearable solution to reproduce the lower-limb motions and map them to the virtual avatar in real time. We employ a single depth camera and design a waist-wearable layout to capture the lower-limb motions relative to the waist. By exploiting the vision data observed by the camera, we further estimate the global velocity of the user.ResultsExperiments are carried out to verify our solution. We quantitatively evaluate the estimated global velocity with an optical motion capture system. We also map the recovered lower-limb motion to the avatar and utilize a standard questionnaire to measure the sense of embodiment. The experiments show that our wearable solution are feasible and effective, being applicable to different people from the perceptual perspective.ConclusionsThe results verify that users are allowed to naturally explore the virtual world with the embodiment using the lightweight equipment.

Development of a Soft Hand and Wrist Rehabilitation Robot Utilizing Lattice Structure Actuators

Theoretical estimation on electrical conductivity, synergy effect and piezoresistive behavior for nanocomposites with hybrid carbon nanotube/graphene based on modified Bethe lattice method