Publications

Relaxed Relational Knowledge Distillation for Inter-Intra Class Learning in Human Activity Recognition

Multiple source domain adaptation in micro-expression recognition

A Mass-Spring Model for Haptic Display of Flexible Object Global Deformation

The global deformation model of virtual object by force is a key issue for haptic interaction between human and virtual reality. A discrete globe mass -spring model is proposed for flexible object deformation, and its surface is divided radially along the force center. The mass-spring system is composed of the nodes connected with radial distributed springs. Using the theory of virtual work, the relations between virtual force and nodal displacements are analyzed to get global deformations. Object globe deformation is simulated by measuring the nodal deformations based on a force equation at each node. According to the model, the deformation of the flexible object is simulated, and synchronously the real-time virtual contact force is provided with delta haptic device.

Development of a Low-Cost Wearable Data Glove for Capturing Finger Joint Angles

Capturing finger joint angle information has important applications in human–computer interaction and hand function evaluation. In this paper, a novel wearable data glove is proposed for capturing finger joint angles. A sensing unit based on a grating strip and an optical detector is specially designed for finger joint angle measurement. To measure the angles of finger joints, 14 sensing units are arranged on the back of the glove. There is a sensing unit on the back of each of the middle phalange, proximal phalange, and metacarpal of each finger, except for the thumb. For the thumb, two sensing units are distributed on the back of the proximal phalange and metacarpal, respectively. Sensing unit response tests and calibration experiments are conducted to evaluate the feasibility of using the designed sensing unit for finger joint measurement. Experimental results of calibration show that the comprehensive precision of measuring the joint angle of a wooden finger model is 1.67%. Grasping tests and static digital gesture recognition experiments are conducted to evaluate the performance of the designed glove. We achieve a recognition accuracy of 99% by using the designed glove and a generalized regression neural network (GRNN). These preliminary experimental results indicate that the designed data glove is effective in capturing finger joint angles.

Scaling analysis and elastic interference of surface responses induced by inclusions in soft matter

Homography-Based Cooperative Teleoperation With Partial Pose Synchronization

Design and Calibration of a New 6 DOF Haptic Device

For many applications such as tele-operational robots and interactions with virtual environments, it is better to have performance with force feedback than without. Haptic devices are force reflecting interfaces. They can also track human hand positions simultaneously. A new 6 DOF (degree-of-freedom) haptic device was designed and calibrated in this study. It mainly contains a double parallel linkage, a rhombus linkage, a rotating mechanical structure and a grasping interface. Benefited from the unique design, it is a hybrid structure device with a large workspace and high output capability. Therefore, it is capable of multi-finger interactions. Moreover, with an adjustable base, operators can change different postures without interrupting haptic tasks. To investigate the performance regarding position tracking accuracy and static output forces, we conducted experiments on a three-dimensional electric sliding platform and a digital force gauge, respectively. Displacement errors and force errors are calculated and analyzed. To identify the capability and potential of the device, four application examples were programmed.

Multi-Constraint Relational Semantic Alignment Towards Image-Text Retrieval

Image-text retrieval aims to align image regions with textual words for semantic matching, facilitating bidirectional retrieval between images and texts. While significant progress has been made in modeling both coarse-grained image-sentence and fine-grained region-word relationships, fully capturing multi granularity correspondences remains a challenge. Many existing methods predominantly depend on region-level segmentation or recognition, which tends to introduce noise, compromise semantic consistency, and increase computational complexity, ultimately limiting retrieval performance. To address these issues, we propose a Multi-constraint Relational Semantic Alignment (McRSA) method, which incorporates three complementary loss-based constraints to enhance multi-granularity alignment while preserving complete information. Specifically, the method includes Posterior Probability Estimation (PPE), which utilizes Bayesian analysis to model causal relationships between image-text feature pairs and labels, reducing intra-class variations for fine-grained alignment. Additionally, a Momentum-driven Centroid Update (MCU) mechanism is introduced to mitigate oscillations and improve modal consistency in coarse-grained representations. A dynamic Feature Scale Adaptation (FSA) module is also employed, adjusting feature scales across modalities to alleviate granularity discrepancies and improve alignment robustness. Extensive experiments on five public datasets (Flickr30K, MS-COCO, RSTPReid, CUHK-PEDES, and ICFG-PEDES) demonstrate that McRSA achieves competitive retrieval performance compared to existing methods. Code and pre-trained models are available at https://github.com/xiaoyiseu/McRSA.

Recognition of Human Inner Emotion Based on Two-Stage FCA-ReliefF Feature Optimization

Currently, there is a growing interesting in emotion recognition. Representation of emotional states is a very challenging issue. Considering the calculation cost and generalization capability for practical application, a series of features which contain common time and frequency domain are extracted from physiological signals to represent different emotional states. To reduce feature dimensionality and improve the emotion recognition accuracy, a two-stage feature optimization method based on feature correlation analysis (FCA) and ReliefF algorithm is proposed to select critical features. Firstly, FCA is employed to analyze the redundancy between features, then ReliefF is adopted to analyze the correlation between features and categories, and the optimal feature subset is obtained using the two-stage FCA-ReliefF feature optimization method. Support vector machine is employed as the classifier to evaluate classification performance in this investigation. The effectiveness of the method which is proposed is validated by testing on two publicly available multimodal emotion datasets, Augsburg Biosignal Toolbox (AuBT) and Database for Emotion Analysis Using Physiological Signals (DEAP). Compared with recent similar reported studies, the method developed in this research for emotion recognition is stable and competitive, and its accuracy reaches to 98.40% (AuBT) and 92.34% (DEAP).

Real-Time Obstacle Avoidance for Telerobotic Systems Based on Equipotential Surface

Redundant manipulators offer a dual advantage of flexibility and dexterity and can be used in many civilian and military areas. However, operating such systems by teleoperation is challenging because of the redundancy and unstructured task environment, which result in the human operator suffering a huge burden when telemanipulator is facing the complicated obstacles. The existing methods usually use some off-line algorithms to solve the problem of obstacle avoidance. It is difficult for them to meet the requirements of real-time teleoperation in some unknown environment. This paper presents an on-line method for a telerobotic system to take advantage of redundancy to avoid obstacle, which is based on real-time sensor information. With this method, the human operator can focus attention on the end-effector operation regardless of the obstacle avoidance of other parts. The effectiveness and advantage of the method are well demonstrated by experiments.

Set–Reset latch logical operation induced by colored noise

Experiments and assessments of a 3-DOF haptic device for interactive operation

Haptic devices have been applied in interactive operation to perform contact tasks. To explore the haptic perception characteristics of typical push-pull and rotation operation, an experimental system was built by incorporating a three degrees of freedom (3-DOF) haptic device and the virtual environment. In this system, the haptic device is used to provide motion commands to control the avatar in the virtual environment and to exert haptic feedback on the human operator generated by three motors. In order to evaluate the main influential factors of interactive system based on haptic devices, ergonomic assessments are designed and experimentally implemented. Preliminary studies on the factors including restoring force, guidance force, speed of the virtual avatar, and the arm length have been conducted. The results are of great significance for the design of a haptic device and haptic interaction system by analyzing the specific requirements of ergonomics.

Research on 3D Virtual Environment Modeling Technology for Space Tele-robot

Abstract To overcome the time delay in tele-robot system, a new 3D virtual environment modeling technology is proposed in this paper. The interactions between slave robot and environment can be attained in advance in a 3D virtual environment in the master side, therefore an accurate operation of space tele-robot can be realized and the effect of time delay would be minimized. Firstly, 3D virtual interaction scenario is modeled as a point cloud data image, and the target objects are recognized from the image and reconstructed by using Random Sample And Consensus (RANSAC) algorithm. Secondly, an effective method is proposed to modify the position of virtual robot and calculate the virtual interactive force between the virtual robot and the virtual objects. Lastly, the experiment is completed with the time delay. The errors are lower (10% F.S.), while virtual interaction force is compared with the real force measured by the force sensor. Experimental results show that this 3D virtual environment modeling method is effective and reliable for space tele-robot control.

Toward Measuring Target Perception: First-Order and Second-Order Deep Network Pipeline for Classification of Fixation-Related Potentials

The topdown determined visual object perception refers to the ability of a person to identify a prespecified visual target. This paper studies the technical foundation for measuring the target-perceptual ability in a guided visual search task, using the EEG-based brain imaging technique. Specifically, it focuses on the feature representation learning problem for single-trial classification of fixation-related potentials (FRPs). The existing methods either capture only first-order statistics while ignoring second-order statistics in data, or directly extract second-order statistics with covariance matrices estimated with raw FRPs that suffer from low signal-to-noise ratio. In this paper, we propose a new representation learning pipeline involving a low-level convolution subnetwork followed by a high-level Riemannian manifold subnetwork, with a novel midlevel pooling layer bridging them. In this way, the discriminative power of the first-order features can be increased by the convolution subnetwork, while the second-order information in the convolutional features could further be deeply learned with the subsequent Riemannian subnetwork. In particular, the temporal ordering of FRPs is well preserved for the components in our pipeline, which is considered to be a valuable source of discriminant information. The experimental results show that proposed approach leads to improved classification performance and robustness to lack of data over the state-of-the-art ones, thus making it appealing for practical applications in measuring the target-perceptual ability of cognitively impaired patients with the FRP technique.

Adaptive logical stochastic resonance in time-delayed synthetic genetic networks

In the paper, the concept of logical stochastic resonance is applied to implement logic operation and latch operation in time-delayed synthetic genetic networks derived from a bacteriophage λ. Clear logic operation and latch operation can be obtained when the network is tuned by modulated periodic force and time-delay. In contrast with the previous synthetic genetic networks based on logical stochastic resonance, the proposed system has two advantages. On one hand, adding modulated periodic force to the background noise can increase the length of the optimal noise plateau of obtaining desired logic response and make the system adapt to varying noise intensity. On the other hand, tuning time-delay can extend the optimal noise plateau to larger range. The result provides possible help for designing new genetic regulatory networks paradigm based on logical stochastic resonance.

Research on Stiffness Display Perception of Virtual Soft Object

In this paper, a novel stiffness display device based on deformable length of elastic element control is presented firstly. The stiffness display device is composed of a thin elastic beam and an actuator to adjust the length of the beam. By controlling the beam length, the stiffness display device can reproduce the stiffness of the virtual object from very soft to hard, so that the human can feel it through fingertip as if he directly touches the virtual object by interacting with the device. Then experiments of human fingertip perception by using this device are carried out. The aims of the experiments are to explore the relations between human perception and differential thresholds, and get the characteristics of human fingertip, including force, times, frequency. At last, some important conclusions are drawn. The results can aid in the process of designing haptic devices.

MRUCT: Mixed Reality Assistance for Acupuncture Guided by Ultrasonic Computed Tomography

Chinese acupuncture practitioners primarily depend on muscle memory and tactile feedback to insert needles and accurately target acupuncture points, as the current workflow lacks imaging modalities and visual aids. Consequently, new practitioners often learn through trial and error, requiring years of experience to become proficient and earn the trust of patients. Medical students face similar challenges in mastering this skill. To address these challenges, we developed an innovative system, MRUCT, that integrates ultrasonic computed tomography (UCT) with mixed reality (MR) technology to visualize acupuncture points in real-time. This system offers offline image registration and real-time guidance during needle insertion, enabling them to accurately position needles based on anatomical structures such as bones, muscles, and auto-generated reference points, with the potential for clinical implementation. In this paper, we outline the non-rigid registration methods used to reconstruct anatomical structures from UCT data, as well as the key design considerations of the MR system. We evaluated two different 3D user interface (3DUI) designs and compared the performance of our system to traditional workflows for both new practitioners and medical students. The results highlight the potential of MR to enhance therapeutic medical practices and demonstrate the effectiveness of the system we developed.

Parameter-induced logical stochastic resonance