Publications

A universal emotion recognition method based on feature priority evaluation and classifier reinforcement

YOLOv10-SCI: single-chip microcomputer defect detection based on the improved YOLOv10 network

A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction

Abstract Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.

RepMobile: A MobileNet-Like Network with Structural Re-Parameterization for Sensor-Based Human Activity Recognition

An Extended Proxy-Based Sliding Mode Control of Pneumatic Muscle Actuators

To solve the problem of controlling an intrinsically compliant actuator, pneumatic muscle actuator (PMA), this paper presents an extended proxy-based sliding mode control (EPSMC) strategy. It is well known that the chattering phenomenon of conventional sliding mode control (SMC) can be effectively solved by introducing a proxy between the physical object and desired position, which results in the so-called proxy-based sliding mode control (PSMC). To facilitate the theoretical analysis of PSMC and obtain a more general form of controller, a new virtual coupling and a SMC are used in our proposed EPSMC. For a class of second-order nonlinear system, the sufficient conditions ensuring the stability and passivity are obtained by using the Lyapunov functional method. Experiments on a real-time PMA control platform validate the effectiveness of the proposed method, and comparison studies also show the superiority of EPSMC over the conventional SMC, PSMC, and PID controllers.

Feature Extraction of Horizontal Plane and Optimization of 3D LiDAR SLAM in Indoor Environments

Haptic Rendering of Neural Radiance Fields

The neural radiance field (NeRF) is attracting increasing attentions from researchers in various fields. While NeRF has produced visually plausible results and found its potential applications in virtual reality, users are only allowed to rotate the camera to observe the scene represented as NeRF. We study the haptic interaction with NeRF models in this paper to enable the experience of touching objects reconstructed by NeRF. Existing haptic rendering algorithms do not work well for NeRF-represented models because NeRF is often noisy. We propose a stochastic haptic rendering method to deal with the collision response between the haptic proxy and NeRF. We validate our method with complex NeRF models and experimental results show the efficacy of our proposed algorithm.

Rethinking Kalman Filters for Motor Brain-Machine Interface: The Fundamental Limitations and A Perspective Shift

The Kalman filter has been introduced to the motor brain-machine interface (BMI) field for over 20 years and remains one of the most widely used models due to its simple and intuitive nature. However, this paper demonstrates that the application of Kalman filters in the BMI field violates the model’s own assumptions and numerous neuroscience principles, resulting in six critical limitations: (1) the observation matrix mapping from kinematics to neural activity causes individual neurons to be modeled independently, ignoring neural population activity; aligned kinematic and neural activity sequences fail to capture preparatory information; (3) the presence of behaviorally irrelevant neural activity causes observation weights to be excessively underestimated; (4) noise in the observation matrix renders posterior covariance estimates biased; (5) Kalman gain computation in observation space leads to repeated noise accumulation and (6) computational inefficiency. All these problems can be resolved through a simple perspective shift: separating the decoder from the Kalman filter and treating its predicted kinematics as the Kalman filter’s observation rather than neural activity. Experiments conducted on CRCNS-recorded monkey dorsal premotor area and primary motor signals performing computer cursor control tasks validate the existence of the six limitations and their violations of neuroscience principles, while demonstrating the superiority of the improved Kalman filter across all evaluated metrics.

Adaptive Finite-Time Control Scheme for Teleoperation With Time-Varying Delay and Uncertainties

The communication time delay and uncertain models of robotic manipulators are the major problem in the teleoperation system, which can reduce the performance and stability of the system. This article proposed a novel finite-time adaptive control scheme for position and force tracking performances of the teleoperation system. First, a combined auxiliary error system with position and force tracking errors is designed. Second, a velocity feedback filter is introduced, and a new auxiliary variable function with finite-time structure is designed for controller design. The radial basis function neural network (RBFNN) is applied to estimate the uncertain parts. Then the finite-time adaptive control scheme and adaptive laws are given. Third, based on the Lyapunov method, stability and finite-time performance are demonstrated. And finally, the simulation and experimental studies (with Phantom Ommi devices) are performed and demonstrate the effectiveness of the proposed control scheme on teleoperation position/force tracking.

Flexible and Wireless Normal‐Tangential Force Sensor Based on Resonant Mechanism for Robotic Gripping Applications

Abstract The ability to detect both normal and tangential forces is essential for robotic tactile systems, while most available sensors implement this function at the cost of complex electrode structure and wiring. Here a fully flexible sensor for wireless detection of normal and tangential force based on resonant mechanism is reported. The components of the sensor are designed and optimized to meet the requirement, and the sensor shows good sensitivity, hysteresis characteristics, response time, and stability in the wired test. Wireless operation is carried out through inductive coupling, and the collected spectrum of reflection coefficient can reflect both normal force and tangential force information. In the case of individual input, the sensitivity is −0.19 MHz kPa −1 to normal force in the range of 0–200 kPa, and −0.031 dB kPa −1 to tangential force in the range of 0–50 kPa. As proofs of concept, maximum static friction coefficient tests and robotic gripping experiments are demonstrated for preliminary determination of surface roughness and material identification respectively. In particular, a total recognition accuracy of 92.5% is achieved by random forest‐based machine learning for balls with different materials in the gripping experiment.

Time delay compensation for nonlinear bilateral teleoperation: A motion prediction approach

This paper addresses the time delay compensation problem for nonlinear teleoperation system. A novel motion prediction approach is proposed based on a state observer with a cascade structure. The actual positions of master robot are estimated on the slave side by using delayed information of measurements. The prediction errors remain bounded under an appropriate set of assumptions for the system uncertainties. Moreover, an essential theorem for slave controller design is proposed to demonstrate performance recovery of the closed-loop system with cascade observer. By using the predictions, the forward time delay is effectively compensated when we design the controller of slave robot. Simulations are performed to verify the effectiveness of proposed method. The prediction results are highlighted through comparing with other two kinds of cascade predictors.

Adaptive Visual-Force Fusion Control for Robotic Phantom Placement in Nuclear Equipment Inspection

Multi-dimensional force sensor for haptic interaction: a review

Haptic interaction plays an important role in the virtual reality technology, which let a person not only view the 3D virtual environment but also realistically touch the virtual environment. As a key part of haptic interaction, force feedback has become an essential function for the haptic interaction. Therefore, multi-dimensional force sensors are widely used in the fields of virtual reality and augmented reality. In this paper, some conventional multi-dimensional force sensors based on different measurement principles, such as resistive, capacitive, piezoelectric, are briefly introduced. Then the mechanical structures of the elastic body of multi-dimensional force sensors are reviewed. It is obvious that the performance of the multi-dimensional force sensor is mainly dependent upon the mechanical structure of elastic body. Furthermore, the calibration process of the force sensor is analyzed, and problems in calibration are discussed. Interdimensional coupling error is one of the main factors affecting the measurement precision of the multi-dimensional force sensors. Therefore, reducing or even eliminating dimensional coupling error becomes a fundamental requirement in the design of multi-dimensional force sensors, and the decoupling state-of-art of the multi-dimensional force sensors are introduced in this paper. At last, the trends and current challenges of multi-dimensional force sensing technology are proposed.

Design of new research platform of telepresence telerobot system

A new design strategy for a research platform of a telepresence telerobot system based on virtual reality technology is put forward. The design frame of the system is simply described, and its important core techniques are described. An octrees data structure is utilized to build kinematic and dynamic modeling of the virtual simulation environment, Delphi+OpenGL+3DS MAX are adopted to carry through the virtual modeling and visible simulation exploitation of the slave-robot and its environment. Photo-correction is adopted to correct positioning deviation of the virtual geometric model and modeling errors. The cost of software and hardware equipment for the research platform realized is low. The master/slave robot (manipulator) system and all software in the system were designed and manufactured by our research group. The performance of the system has reached the level required for research. An indispensable experiment base is provided for the research of a telepresence telerobot system based on virtual reality technology.

A miniature flexible sampler for subsurface lunar exploration

Design and Control of a Fully Actuated Aerial Manipulator System for Measuring the Thickness of Metal Facilities

Underwater multilateral tele-operation control with constant time delays

Model-based linear control of nonlinear pneumatic soft bending actuators

Abstract Advanced model-based control techniques hold great promise for the precise control of pneumatic soft bending actuators (PSBAs) with strong nonlinearities. However, most previous controllers were designed in a cumbersome nonlinear form. Considering the simplicity of linear system theory, this paper presents a novel perspective on using model-based linear control to handle nonlinear PSBAs, and for the first time, summarizes two methodologies, global linearization and pseudo-linear construction. Derived from them, Koopman-based and hysteresis-based linear control approaches are proposed, respectively. For the former, a novel fusion prediction equation is developed to build a high-fidelity Koopman model, realizing global linearization, and then the linear model predictive control (MPC) is deployed. For the latter, the inverse of the generalized Prandtl–Ishlinskii (GPI) model cascades with the PSBA to construct a pseudo-linear system, eliminating the asymmetric hysteresis, which activates the linear proportional-integral-derivative (PID) control. It is worth noting that the above two are based on data-driven models adapted to various PSBAs with material and structural customization. Finally, the two model-based linear control approaches are verified and compared through a series of experiments. The results show that the proposed linear controls, with more concise designs, achieve comparable or even superior performance than nonlinear controls.