Publications

Chinese acupuncture practitioners primarily depend on muscle memory and tactile feedback to insert needles and accurately target acupuncture points, as the current workflow lacks imaging modalities and visual aids. Consequently, new practitioners often learn through trial and error, requiring years of experience to become proficient and earn the trust of patients. Medical students face similar challenges in mastering this skill. To address these challenges, we developed an innovative system, MRUCT, that integrates ultrasonic computed tomography (UCT) with mixed reality (MR) technology to visualize acupuncture points in real-time. This system offers offline image registration and real-time guidance during needle insertion, enabling them to accurately position needles based on anatomical structures such as bones, muscles, and auto-generated reference points, with the potential for clinical implementation. In this paper, we outline the non-rigid registration methods used to reconstruct anatomical structures from UCT data, as well as the key design considerations of the MR system. We evaluated two different 3D user interface (3DUI) designs and compared the performance of our system to traditional workflows for both new practitioners and medical students. The results highlight the potential of MR to enhance therapeutic medical practices and demonstrate the effectiveness of the system we developed.

The topdown determined visual object perception refers to the ability of a person to identify a prespecified visual target. This paper studies the technical foundation for measuring the target-perceptual ability in a guided visual search task, using the EEG-based brain imaging technique. Specifically, it focuses on the feature representation learning problem for single-trial classification of fixation-related potentials (FRPs). The existing methods either capture only first-order statistics while ignoring second-order statistics in data, or directly extract second-order statistics with covariance matrices estimated with raw FRPs that suffer from low signal-to-noise ratio. In this paper, we propose a new representation learning pipeline involving a low-level convolution subnetwork followed by a high-level Riemannian manifold subnetwork, with a novel midlevel pooling layer bridging them. In this way, the discriminative power of the first-order features can be increased by the convolution subnetwork, while the second-order information in the convolutional features could further be deeply learned with the subsequent Riemannian subnetwork. In particular, the temporal ordering of FRPs is well preserved for the components in our pipeline, which is considered to be a valuable source of discriminant information. The experimental results show that proposed approach leads to improved classification performance and robustness to lack of data over the state-of-the-art ones, thus making it appealing for practical applications in measuring the target-perceptual ability of cognitively impaired patients with the FRP technique.

During the past decade, deep convolutional neural networks (CNNs) have shown remarkable success in ubiquitous human activity recognition (HAR) scenarios. However, prior most works are static, which have to manually predefine a fixed kernel size before training, hence requiring time-consuming human intervention. A more efficient solution is to determine the kernel size during the training stage, rather than the design stage. Different from previous static designs, this article proposes an adaptive solution called dynamic Gaussian convolution (DgConv), which can adaptively learn an optimal kernel size over sensor data for each individual convolutional layer, without requiring laborious human intervention. We conduct extensive experiments on several popular benchmark HAR datasets, including PAMAP2, UCI-HAR, UNIMIB-SHAR, OPPORTUNITY, HAG, and WISDM. The results show that the proposed DgConv can consistently enhance model performance while maintaining similar inference-time costs, which shows a notable performance improvement compared to static convolution. In addition, our approach may improve the robustness of deep CNN classifiers to various types of noise sources. A practical implementation is evaluated on a real mobile device. In summary, our DgConv could benefit the neural architecture search for activity recognition by narrowing the search space by excluding kernel size, which will be explored in future work.

Aiming at the problem of low recognition accuracy of motor imagery electroencephalogram signal due to individual differences of subjects, an individual adaptive feature representation method of motor imagery electroencephalogram signal is proposed in this paper. Firstly, based on the individual differences and signal characteristics in different frequency bands, an adaptive channel selection method based on expansive relevant features with label F (ReliefF) was proposed. By extracting five time-frequency domain observation features of each frequency band signal, ReliefF algorithm was employed to evaluate the effectiveness of the frequency band signal in each channel, and then the corresponding signal channel was selected for each frequency band. Secondly, a feature representation method of common space pattern (CSP) based on fast correlation-based filter (FCBF) was proposed (CSP-FCBF). The features of electroencephalogram signal were extracted by CSP, and the best feature sets were obtained by using FCBF to optimize the features, so as to realize the effective state representation of motor imagery electroencephalogram signal. Finally, support vector machine (SVM) was adopted as a classifier to realize identification. Experimental results show that the proposed method in this research can effectively represent the states of motor imagery electroencephalogram signal, with an average identification accuracy of (83.0±5.5)% for four types of states, which is 6.6% higher than the traditional CSP feature representation method. The research results obtained in the feature representation of motor imagery electroencephalogram signal lay the foundation for the realization of adaptive electroencephalogram signal decoding and its application.针对运动想象脑电信号因受试者个体差异导致辨识精度低的问题，本文提出了基于个体自适应的运动想象脑电信号特征表征方法。首先从个体差异和频带信号特点出发，提出了基于拓展式相关特征（ReliefF）的自适应通道选择方法；通过提取各频带信号5个时频域观察特征，运用ReliefF算法对各频带信号通道进行有效性评估，进而实现各频带信号通道选择。其次提出了基于快速相关滤波算法（FCBF）的共空间模式（CSP）特征表征方法（CSP-FCBF）；通过CSP提取脑电信号特征，运用FCBF进行特征优化得到最优特征集，从而实现运动想象脑电信号状态有效表征。最后使用支持向量机（SVM）作为分类器进行分类辨识。实验分析结果表明，本文所提方法能有效实现运动想象脑电信号状态表征，四类状态平均辨识精度达到了（83.0±5.5）%，较传统的CSP特征表征方法提高6.6%。本文在运动想象脑电信号特征表征方面取得的研究成果，为实现自适应的脑电信号解码及其应用奠定了基础。.

Deep convolutional networks have recently achieved significant success in sensor-based human activity recognition (HAR). Existing works are mainly devoted to extracting multiscale activity features from sensor time series by increasing network depth or width, which are not friendly to mobile devices with limited computing resources. It still remains a challenging issue to strike an ideal trade-off between activity recognition performance and inference-time costs <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., the latency and memory footprint. To address this issue, we propose a diverse-branch convolution (DBC) scheme, which could strengthen the representation capacity of vanilla convolution via exploiting diverse branches of different scales and complexities to enrich activity feature space. Then DBC could be equivalently transformed into a single convolution layer after training for HAR deployment. In such manner, DBC only complicates training-time microstructure while preserving inference-time macrostructure, where the model performance can be boosted to a higher level and then converted back to its original inference-time structure for activity recognition. Extensive experiments are conducted on three publicly available benchmark datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., OPPORTUNITY, UniMiB-SHAR, WISDM, and a self-collected Weakly Labeled HAR dataset, which verify that our method can give a lift in performance while maintaining the same number of parameters and FLOPs as baselines. Finally, actual inference-time cost is evaluated for HAR deployment on a Raspberry Pi platform.

Brain-controlled robotic hand rehabilitation systems based on motor intention recognition have been used to promote recovery of hand function in stroke patients. However, the low decoding accuracy of motor imagery (MI) and unclear neural response mechanisms limit its widespread application. This study introduces a novel vibrotactile-assisted brain-controlled soft robotic hand rehabilitation system to validate its effectiveness in activating the motor sensory areas of the brain and to explore the neural response mechanisms of vibration stimulation in hand rehabilitation training. A total of 23 healthy subjects and 5 stroke patients were recruited to perform EEG and fNIRS-based experiments. Healthy subjects performed an EEG-based active rehabilitation task and an fNIRS-based passive rehabilitation task driven by the soft glove. Stroke patients only completed an EEG-based passive rehabilitation task. All experiments were conducted under two conditions: with and without vibrotactile stimulation. EEG results revealed that vibration stimulation significantly enhanced motor-sensory cortex activation during MI, and improved the online decoding performance of subjects with poor training outcomes. Grasping and stretching movements driven by the soft glove effectively activated the subjects' motorsensory cortex. Vibration stimulation boosted the event-related desynchronization (ERD) phenomenon in the contralateral somatosensory cortex of the healthy subjects, but was not significant in the motor cortex. Meanwhile, it strengthened bilateral sensorimotor activation in stroke patients. Moreover, fNIRS results indicated that vibration stimulation increased the concentration of HbO in the motor-sensory areas during passive movement and enhanced the bidirectional functional connectivity between the left and right hemispheres. These findings suggest that the proposed tactile-assisted hand rehabilitation system can effectively enhance neural activation in the motor-sensory cortex, potentially leading to improved hand function recovery in stroke patients.

Abstract High degrees of freedom (DOF) mobile manipulators provide more flexibility than conventional manipulators. They also provide manipulation operations with a mobility capacity and have potential in many applications. However, due to high redundancy, planning and control become more complicated and difficult, especially when obstacles occur. Most existing obstacle avoidance methods are based on off-line algorithms and most of them mainly focus on planning a new collision-free path, which is not appropriate for some applications, such as teleoperation and uses many system resources as well. Therefore, this paper presents an online planning and control method for obstacle avoidance in mobile manipulators using online sensor information and redundancy resolution. An obstacle contour reconstruction approach employing a mobile manipulator equipped with an active laser scanner system is also introduced in this paper. This method is implemented using a mobile manipulator with a seven-DOF manipulator and a four-wheel drive mobile base. The experimental results demonstrate the effectiveness of this method.

Surface electromyography (sEMG) signals are widely used in the recognition of hand gestures. Nowadays, researchers usually increase the number of sEMG signal measurement positions and extract multiple features to improve the recognition accuracy. In this paper, we propose a sEMG measurement position and feature optimization strategy for gesture recognition based on Analysis of Variance (ANOVA) and neural networks. Firstly, four channels of raw sEMG signals are acquired, and four time-domain features are extracted. Then different neural networks are trained and tested by using different data sets which are obtained based on the combination of different measurement positions and features. Finally, ANOVA and Tukey HSD testing are conducted based on the gesture recognition results of different neural networks. We obtain the optimal measurement position sets for gesture recognition when different feature sets are used, and similarly, the optimal feature sets when different measurement position sets are used. Our experimental results show that the feature set of zero crossing and integrated sEMG provides the highest gesture recognition accuracy, which is 94.83%, when four channels of sEMG signals are used; the optimal measurement position set when four sEMG signal features are used for hand gesture recognition is P1+P3+P4, which provides an accuracy of 94.6%.