Publications

Joint Design, Safety Margin, and Ground Experiments of an Underactuated Asteroid Lander

Learn From Others and Be Yourself in Federated Human Activity Recognition via Attention-Based Pairwise Collaborations

Federated learning has recently been an emerging learning paradigm for training deep neural networks for activity recognition on resource-limited portable devices such as smartphones. However, prior most works typically focus on training a single global model while ignoring potential intrinsic relationship between clients, which cannot generalize well in heterogeneous federated human activity recognition (HAR) scenario. To address this issue, this article proposes an alternative via attention-based pairwise collaborations, where each client can collaborate with other similar clients to generate a stronger personalized model per client-specific objectives. Different from previous most works, instead of aggregating a single model, we adopt an attention method to calculate an optimal weighted model combination for each client, based on figuring out to what extent a client could benefit from other clients. Extensive experiments on four public HAR benchmark datasets with non-IID data setting show that our method significantly outperforms several popular state-of-the-art federated learning baselines in terms of overall accuracy while saves more than 50% communication overhead. Several ablation analyses are conducted to provide a deeper insight into the proposed algorithm in federated learning scenario. In particular, we analyze how the pairwise collaborations iteratively encourage similar clients to have stronger collaboration than inconsistent clients with heterogeneous activity data. An actual implementation is evaluated in a network of resource-limited Internet-of-Things (IoT) devices.

A Neuromorphic Approach for Brain-Machine Interface Using Spiking Neural Networks

Brain-machine interfaces (BMIs) have emerged as a promising technology for restoring motor function in paralyzed individuals through direct neural control of prosthetic devices. While conventional decoding algorithms have achieved considerable success, they often overlook the fundamental biological properties of neural information processing. This paper presents a novel approach using Spiking Neural Networks (SNNs), a neuromorphic computing paradigm that closely mimics biological neural dynamics through event-driven processing and spike-timing-dependent plasticity. A SNN-based decoder was implemented for offline decoding of intracortical neural recordings from the primary motor cortex (M1) and dorsal premotor cortex (PMd) to continuous 2D cursor movements in a macaque monkey. This approach leverages the temporal processing capabilities of SNNs to capture the complex, time-varying nature of neural representations, potentially enabling more naturalistic and adaptive BMI control.

Modeling and experimental validation of sawing based lander anchoring and sampling methods for asteroid exploration

An RGBD Fused Real Time Defect Detection and Size Estimation Method for Sewer Robots

Development of a Novel Leader Device with Haptic Feedback for Vascular Interventional Surgery

Robot-assisted vascular interventional surgery has become a hot topic in the field of medical science and engineering. In recent years, the research of robot-assisted vascular interventional surgery has achieved promising results. However, improving the human-robot collaboration performance during the surgery is still a major challenge. To address this challenge, a novel leader device was designed and developed. In detail, the structural design of the leader device fully considers ergonomic design, which can greatly reduce the operating difficulty for surgeons while allowing them to take advantage of their traditional surgical experience. The leader device can recognize the surgeon's operational behaviors, mainly including pushing, pulling, and rotating. Besides, haptic feedback function is realized and provided during the operation, which can help the surgeon accurately grasp the dynamics of the operation and effectively improve the presence and safety of the operation. To verify the effectiveness of the haptic feedback of the leader device, a verification experiment was conducted between the electric current and the force/torque when the angle between the operating rod and the rhombus structure was 30 degrees. The results indicated that the developed leader device can provide the haptic feedback to the surgeon. The leader device with haptic feedback has the potential to improve the human-robot collaboration performance during the operation.

Design and implementation of ZigBee based gateway for environmental monitoring system

This paper proposed a three-part framework for deploying an environmental monitoring system (EMS). An enhanced gateway which supports Bluetooth based local wireless access and GPRS based remote communication is designed to be an interface between the client tier and a deployed ZigBee wireless sensor network. The binding mechanism which associates a service on one node with a service on another node is used so as to allow communication between two services to be performed without the need to specify the address. An implementation for verifying the EMS including the gateway as well as the sMotes is presented, and experimental results show that, the proposed architecture can achieve the environmental monitoring performance, and except the gateway, other nodes in the system work on sleep mode at most of the time, which will improve the lifetime of the whole system efficiently.

Neural Network Ensemble With Evolutionary Algorithm for Highly Imbalanced Classification

Imbalanced data is a major challenge in classification tasks. Most classification algorithms tend to be biased toward the samples in the majority class but fail to classify the samples in the minority class. Recently, ensemble learning, as a promising method, has been rapidly developed in solving highly imbalanced classification. However, the design of the base classifier for the ensemble is still an open question because the optimization problem of the base classifier is gradientless. In this study, the evolutionary algorithm (EA) technique is adopted to solve a wide range of optimization design problems in highly imbalanced classification without gradient information. A novel EA-based classifier optimization design method is proposed to optimize the design of multiple base classifiers automatically for the ensemble. In particular, an EA method with a neural network (NN) as the base classifier termed NN ensemble with EA (NNEAE) is developed for highly imbalanced classification. To verify the performance of NNEAE, extensive experiments are designed for testing. Results illustrate that NNEAE outperforms other compared methods.

Mosquito-inspired active tactile perception for indoor navigation and escape of a rigid-soft coupling blimp robot

For autonomous navigation in indoor environments, aerial robots mostly take cameras, LiDAR, ultrasonic ranging sensors, and other devices for collision avoidance, rarely using tactile sensors. However, cameras are susceptible to lighting conditions, LiDAR requires high computational resources, and ultrasonic sensors have blind zones when measuring at short distances. In contrast, insects and rodents can perceive their surroundings via tactile sensing even in complete darkness. Inspired by the escape strategy of mosquitoes that navigate along boundaries using tactile sensing in confined spaces, this paper proposes an indoor navigation and escape method based on active tactile perception for a blimp robot. The robot comprises a rigid multi-rotor structure and a soft balloon body, with bio-inspired whisker sensors mounted on the soft body surface to enable safe contact with walls. First, we studied mosquito escape behavior experimentally. Then, we designed the robot's mechanical structure and tactile perception system. Subsequently, an interaction model between the robot and the wall was established, and a flight controller was developed. We classified the typical indoor wall scenarios and proposed a 'Sense-Plan-Act' framework for wall-following navigation and escape. Next, the designed controller and strategy were validated through simulations. Finally, we conducted experiments using a robot prototype to verify the proposed method. Results showed that the robot successfully achieved indoor wall-following navigation during flight and ultimately escaped. The proposed active tactile perception method is straightforward and practical for the indoor navigation and escape tasks of blimp robots.

An Intuitively Derived Decoupling and Calibration Model to the Multiaxis Force Sensor Using Polynomials Basis

Precise decoupling and calibration of multiaxis force sensor (MFS) is crucial in engineering applications. This work presents a novel nonlinear decoupling and calibration approach to meet the physical coupling characteristics of the structural strain-deformation transducers. It deals with the most force–voltage responses by the linear prime modeling and the gross error deviation by the nonlinear error modeling. Such a prime-error framework is naturally derived from the conventional least-square (LS) decoupling model with the delicate nonlinear error modeling by the multivariate Bernstein polynomials. The two- and three-axis force sensors are tested and compared with the proposed Bernstein-based prime-error model (BPEM), the LS decoupling model (LSM), and error-based neural network (eNN) learning model, extreme learning machine (ELM), as well as the error-based support vector machine (e-SVM), the interpretable nonlinear decoupling model (IND). Results demonstrate that the proposed BPEM provides an accurate, practical, and effective scheme for modeling and calibrating MFSs.

A high-fidelity virtual liver model incorporating biological characteristics

Flexible tissue modeling plays an important role in the field of telemedicine. It is related to whether the soft tissue deformation process can be accurately, real-time and vividly simulated during surgery. However, most existing models lack the unique biological characteristics. To solve this problem, we proposed a high-fidelity virtual liver model incorporating biological characteristics, such as the viscoelastic, anisotropic and nonlinear biological characteristics. Besides, to the best of our knowledge, our study is the first to introduce the viscoplasticity of biological tissues to improve the fidelity of the liver model. This mothod was proposed to describe the viscoplastic characteristics of the diseased liver resection process, when the liver is in a state of excessive deformation and loss of elasticity, however, there are few works focusing on this problem. The 3DMax2020 and OpenGL4.6 were used to build a liver surgery simulation platform, and the PHANTOM OMNI manual controller was used to sense the feedback force during the operation. The proposed model was verified from three aspects of accuracy, fidelity and real-time performance. The experimental results show that the proposed virtual liver model can enhance visual perception ability, improve deformation accuracy and fidelity.

Wearable Inertial and Pressure Sensors-Based Chest Compression Quality Assessment to Improve Accuracy and Robustness

Chest compression (CC) quality is essential for saving patients’ lives in cardiopulmonary resuscitation. This article presents a wearable CC quality assessment system integrating an inertial measurement unit (IMU) and a flexible pressure sensor. We proposed a CC depth (CCD) detection method that utilizes the spatiotemporal relationship between the pressure and depth to determine key timestamps and corrects the integration error by mapping the timestamps to velocity to exclude empty strokes. The method improves the CCD calculation accuracy of traditional acceleration dual integration algorithms. We also calculated the CC rate (CCR) based on the time points of the extremum of the pressure signal and evaluated the CC balance (CCB) by segmenting the pressure signal. Moreover, we used the residual pressure after CC release and the maximum pressure as observation indicators to evaluate the recoil adequacy adapting to different sternum rigidity. Furthermore, we conducted time- and frequency-domain analyses on the pitch angle signals and established a support vector machine (SVM)-based classifier to recognize incorrect arm and elbow postures. We performed CC experiments using the designed system and a manikin and constructed four datasets. We validated the proposed methods using the datasets for the multidimensional CC quality indicator evaluations. The absolute median errors of CCD and CCR detections were 1.17 mm and 0.11 times/min, respectively. The results indicated that our system and methods have high detection accuracy and robustness for future clinical applications.

Decoding Natural Grasping Behaviors: Insights Into MRCP Source Features and Coupling Dynamics

The effective decoding of natural grasping behaviors is crucial for the natural control of neural prosthetics. This study aims to investigate the decoding performance of movement-related cortical potential (MRCP) source features between complex grasping actions and explore the temporal and frequency differences in inter-muscular and cortical-muscular coupling strength during movement. Based on the human grasping taxonomy and their frequency, five natural grasping motions-medium wrap, adducted thumb, adduction grip, tip pinch, and writing tripod-were chosen. We collected 64-channel electroencephalogram (EEG) and 5-channel surface electromyogram (sEMG) data from 17 healthy participants, and projected six EEG frequency bands into source space for further analysis. Results from multi-classification and binary classification demonstrated that MRCP source features could not only distinguish between power grasp and precision grasp, but also detect subtle action differences such as thumb adduction and abduction during the execution phase. Besides, we found that during natural reach-and-grasp movement, the coupling strength from cortical to muscle is lower than that from muscle to cortical, except in the hold phase of γ frequency band. Furthermore, a 12-Hz peak of inter-muscular coupling strength was found in movement execution, which might be related to movement planning and execution. We believe that this research will enhance our comprehension of the control and feedback mechanisms of human hand grasping and contributes to a natural and intuitive control for brain-computer interface.

Deep Ensemble Learning for Human Activity Recognition Using Wearable Sensors via Filter Activation

During the past decade, human activity recognition ( HAR ) using wearable sensors has become a new research hot spot due to its extensive use in various application domains such as healthcare, fitness, smart homes, and eldercare. Deep neural networks, especially convolutional neural networks ( CNNs ), have gained a lot of attention in HAR scenario. Despite exceptional performance, CNNs with heavy overhead is not the best option for HAR task due to the limitation of computing resource on embedded devices. As far as we know, there are many invalid filters in CNN that contribute very little to output. Simply pruning these invalid filters could effectively accelerate CNNs , but it inevitably hurts performance. In this article, we first propose a novel CNN for HAR that uses filter activation. In comparison with filter pruning that is motivated for efficient consideration, filter activation aims to activate these invalid filters from an accuracy boosting perspective. We perform extensive experiments on several public HAR datasets, namely, UCI-HAR ( UCI ), OPPORTUNITY ( OPPO ), UniMiB-SHAR ( Uni ), PAMAP2 ( PAM2 ), WISDM ( WIS ), and USC-HAD ( USC ), which show the superiority of the proposed method against existing state-of-the-art ( SOTA ) approaches. Ablation studies are conducted to analyze its internal mechanism. Finally, the inference speed and power consumption are evaluated on an embedded Raspberry Pi Model 3 B plus platform.

A novel miniature multi-mode haptic pen for image interaction on mobile terminal

In this paper, a novel miniature multi-mode haptic pen is developed to interact with image on mobile terminal. The haptic pen can provide two types of vibrotactile feedback by linear resonant actuator (LRAs) and piezo-ceramic actuator. It can provide passive force feedback by a miniature magnetorheological (MR) fluids damper which is designed independently. In order to obtain the features of an image, the haptic interactive software is developed based on Android system. The software can extract the height of each pixel by the shape from shading (SFS) algorithm, and it can also detect the edges of an image by Sobel algorithm. When the haptic pen slides on an image, feature information of the image will be transmitted to the haptic pen via Bluetooth, including the height of each pixel, the edges of an image and the stiffness of virtual objects. Meanwhile, user will get multi-mode haptic sensation by different patterns of haptic display methods. Finally, three haptic interaction experiments are carried out to evaluate the performance of the haptic pen in displaying the height, contour and stiffness of an image.

MDDP: Making Decisions From Different Perspectives in Multiagent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) has made remarkable progress in recent years. However, in most MARL methods, agents share a policy or value network, which is easy to result in similar behaviors of agents and thus limits the flexibility of the method to handle complex tasks. To enhance the diversity of agent behaviors, we propose a novel method, Making Decisions from Different Perspectives (MDDP). This method enables agents to switch flexibly between different policy roles and make decisions from different perspectives, which can improve the adaptability of policy learning in complex scenarios. Specifically, in MDDP, we design a new Self-attention and Gated Recurrent Unit (GRU) based Dueling Architecture Network (SG-DAN) to estimate the individual <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> -values. SGDAN contains two components: the new Self-Attention based Role-switching network (SAR) and the capable GRU-based State value Estimation network (GSE). SAR takes charge of action advantage estimation and GSE is responsible for state value estimation. Experimental results on the challenging StarCraft II micromanagement benchmark not only verify the modeling reasonability of MDDP but also demonstrate its performance superiority over the related advanced approaches.

Improved Haptic Rendering through Suppressing the Position-Sensor Quantization Effects

Rendering rigid virtual objects remains a challenging problem in the field of haptics. The "nonidealities" due to time discretization and position quantization are the main factors that constrain the maximum achievable stiffness for the virtual objects. While the former can be addressed with many methods, the latter becomes a critical problem. New simulation methods, smoothing the output force with a dynamic moving-average filter (MAF) while the velocity is low, and a nonlinear spring model combining the linear spring model with a square-root spring model, are presented to suppress the noisy behavior of position-sensor quantization. The proposed methods are testified experimentally with ldquovirtual wallrdquo prototype system based on Delta haptic device.

MPC Design of a Continuum Robot for Pulmonary Interventional Surgery using Koopman Operators