Publications

Although the electroencephalography (EEG) based brain–computer interface (BCI) has been successfully developed for rehabilitation and assistance, it is still challenging to achieve continuous control of a brain-actuated mobile robot system. In this study, we propose a continuous shared control strategy combining continuous BCI and autonomous navigation for a mobile robot system. The weight of shared control is designed to dynamically adjust the fusion of continuous BCI control and autonomous navigation. During this process, the system uses the visual-based simultaneous localization and mapping (SLAM) method to construct environmental maps. After obtaining the global optimal path, the system utilizes the brain-based shared control dynamic window approach (BSC-DWA) to evaluate safe and reachable trajectories while considering shared control velocity. Eight subjects participated in two-stage training, and six of these eight subjects participated in online shared control experiments. The training results demonstrated that naïve subjects could achieve continuous control performance with an average percent valid correct rate of approximately 97% and an average total correct rate of over 80%. The results of online shared control experiments showed that all of the subjects could complete navigation tasks in an unknown corridor with continuous shared control. Therefore, our experiments verified the feasibility and effectiveness of the proposed system combining continuous BCI, shared control, autonomous navigation, and visual SLAM. The proposed continuous shared control framework shows great promise in BCI-driven tasks, especially navigation tasks for brain-driven assistive mobile robots and wheelchairs in daily applications.

Assist-as-needed (AAN) control strategy is applied in robot-aided therapy to promote the subject's engagement with minimal assistance. Current methods are based on physical measurements or employ a model to estimate the subject's engagement, which indirectly reflects the physiology of the subject or needs a complicated modeling process. Besides, manual adjustments are required to tune the parameters of the AAN controller to adapt to the estimated engagement of subjects. Such two drawbacks of previous studies have limited their effectiveness for promoting the subject's engagement during the training. In this paper, we propose an engagement enhancement method based on Bayesian Optimization for the adaptive AAN controller. Firstly, we evaluate the subject's engagement by using sEMG-based muscle activation, which directly relates to the physiology of the subject and eliminates the need for complicated modeling of engagement. Next, Bayesian optimization, which is tolerant of noisy measurements and human adaptation, is used to search for the optimal parameter of the AAN controller for each training trial. Finally, we conduct a study with sixteen healthy human subjects to verify the effectiveness of the proposed method. The experimental results showed that subjects' engagement in the experiment group improved more significantly and was maintained at a higher level than that in the control group during the training. The performance improvement after the training obtained in the experiment group was higher than that in the control group (45.42% v.s. 30.40%). The experimental results indicate that the proposed method may be promising for transferring to the rehabilitation of post-stroke patients.

This paper studies a novel application of an aerial manipulator (AM)-the contact-based landing on a mobile platform. An AM is inherently unstable, under-actuated, and usually loses some DOFs while contacting environments. Meanwhile, the AM's flight state is susceptible to uncertain movements of the mobile platform, such as acceleration, sudden stopping, and reversing. To accomplish the contact-based landing mission, a robust controller is first designed to maintain a steady contact-based flight. Then a hierarchical control framework is applied, integrating the controllers in free-flight and restricted-flight stages. An AM and a mobile platform are developed for contact-based flight experiments. The proposed scheme is reliable and has good repeatability in experiments. To the best of our knowledge, this is the first time an AM has been implemented to conduct a contact-based landing, which is also an innovative landing approach for rotorcraft UAVs.

Abstract A novel concept—the contact-based landing on a mobile platform—is proposed in this paper. An adaptive backstepping controller is designed to deal with the unknown disturbances in the interactive process, and the contact-based landing mission is implemented under the hybrid force/motion control framework. A rotorcraft aerial vehicle system and a ground mobile platform are designed to conduct flight experiments, evaluating the feasibility of the proposed landing scheme and control strategy. To the best of our knowledge, this is the first time a rotorcraft unmanned aerial vehicle has been implemented to conduct a contact-based landing. To improve system autonomy in future applications, vision-based recognition and localization methods are studied, contributing to the detection of a partially occluded cooperative object or at a close range. The proposed recognition algorithms are tested on a ground platform and evaluated in several simulated scenarios, indicating the algorithm’s effectiveness.

In recent years, there has been a growing research interest in using deep learning to resolve the issue of electroencephalogram (EEG)-based emotion recognition. Current research emphasizes exploiting the useful information from each single EEG channel or each individual set of multichannel EEG, but overlooks the correlation information among different multichannel EEG sets. To explore such discriminative correlation information, we propose a novel and effective method, "trainable adjacency relation driven graph convolutional network (TARDGCN)," which contains two complementary modules: 1) trainable adjacency relation (TAR) and 2) graph convolutional network (GCN). TAR optimizes the local pairwise positions of multichannel EEG sets, which helps form an improved graphic representation for GCN to learn the global correlation among these sets for classification. The proposed method is capable of dealing with the problem of small sample size but large data variation in this issue. Our experimental results conducted on the databases DREAMER and DEAP in the subject-dependent and subject-independent modes show that TARDGCN outperforms the state-of-the-art approaches in classifying all of valence, arousal, and dominance.

Most of the current power line inspection robots use cameras and LiDARs to inspect the power line surfaces and the surrounding environment. But it is still difficult to detect the internal defects of the power lines. In this letter, the design and implementation of LineSpyX, a novel power line inspection robot based on digital radiography (DR), is introduced to solve the problem of non-destructive testing (NDT) of the overhead Aluminum Conductor Composite Core (ACCC) wires. The proposed robot has a stable wrapped mechanical structure with a moving system, a live work system, and a NDT system. The wheeled moving system enables the robot to move on the wires and cross obstacles such as vibration dampers. The NDT system consists of a portable X-ray generator and a DR detection panel. When the robot performs the inspection task, the X-ray goes up through the ACCC wire to the panel, where the X-ray images of the internal carbon fiber cores are recorded. A deep learning based defect diagnosis method combined with manual diagnosis is proposed to detect potential defects. The main functionalities of the developed robot are verified by lab experiments and field tests.

Noninvasive vital sign (e.g., respiration and heart-beat) monitoring via millimeter wave radar (mmWave) presents a promising paradigm for health management and disease diagnostics, yet the widespread adoption of this technology is often hindered by motion artifacts, environmental noise, and limited adaptability to diverse scenarios. This paper proposes a robust respiratory and heartbeat measurement approach that integrates enhanced radar sensing with adaptive decomposition algorithms for diverse healthcare environments. Initially, we introduce a target-signal cooperative mechanism that combines target detection for spatial localization with beamforming-driven directional adjustments and channel coherent accumulation, thereby enhancing the signal to noise ratio (SNR) in the region of interest. Then, the marine predators algorithm (MPA) is implemented to optimize variational mode decomposition (VMD) parameters for precise vital sign extraction. Prior to this, a cascaded signal processing pipeline is applied, comprising static clutter suppression, differentiation and cross multiplication (DACM) phase extraction, and signal smoothing to mitigate noise effects. Experimental validation encompasses two configurations: (1) multi-angle and multi-distance static indoor scenarios, (2) four motion scenarios with controlled body movements, designed to emulate real-world conditions. The experimental results indicate superior respiratory rate (RR) estimation performance, achieving a mean absolute error (MAE) of 1.02 bpm. For heartbeat rate (HR) monitoring, MAEs of 3.23 bpm (static scenarios, 95.78% accuracy) and 4.16 bpm (lightweight motion scenarios, 94.88% accuracy) demonstrate significant feasibility and substantial reliability.