Publications

Multiagent reinforcement learning (MARL) methods have good application performances and prospects in cooperative tasks. To improve the capability of agent policy learning in new scenarios, some methods transfer the learned policy knowledge to new scenarios. However, most methods only focus on the knowledge transfer of individual agent policies, neglecting the credit assignment among agents in cooperative tasks, which results in a transfer bias of cooperative policies. In this paper, we propose a novel method, transferable policy generation and credit assignment (TPGCA) network for cooperative MARL. TPGCA can transfer the entire MARL model by the constructed transferable <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula>-value network and mixing network. Specifically, in TPGCA, to enhance the effectivity and transferability of agent policies, we design the correspondence network between observations and actions (COA) on the basis of transformer and gated recurrent unit (GRU). To implement the reliable credit assignment and diminish the transfer bias, we devise the role-based joint <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula>-value decomposition network (RVD) that can evaluate the contributions of agents from different observation perspectives. Experimental results in various micro-management scenarios on StarCraft multiagent challenge (SMAC) and multiagent particle environment (MPE) sufficiently demonstrate the effectiveness and transferability of TPGCA.

The Kalman filter has been introduced to the motor brain-machine interface (BMI) field for over 20 years and remains one of the most widely used models due to its simple and intuitive nature. However, this paper demonstrates that the application of Kalman filters in the BMI field violates the model’s own assumptions and numerous neuroscience principles, resulting in six critical limitations: (1) the observation matrix mapping from kinematics to neural activity causes individual neurons to be modeled independently, ignoring neural population activity; aligned kinematic and neural activity sequences fail to capture preparatory information; (3) the presence of behaviorally irrelevant neural activity causes observation weights to be excessively underestimated; (4) noise in the observation matrix renders posterior covariance estimates biased; (5) Kalman gain computation in observation space leads to repeated noise accumulation and (6) computational inefficiency. All these problems can be resolved through a simple perspective shift: separating the decoder from the Kalman filter and treating its predicted kinematics as the Kalman filter’s observation rather than neural activity. Experiments conducted on CRCNS-recorded monkey dorsal premotor area and primary motor signals performing computer cursor control tasks validate the existence of the six limitations and their violations of neuroscience principles, while demonstrating the superiority of the improved Kalman filter across all evaluated metrics.

During recent years, human activity recognition (HAR) using smart wearable sensors has become a main research focus in ubiquitous computing scenario. Deep convolutional neural networks (CNNs) have achieved significant success in HAR due to their automatic feature extracting ability in capturing local activity details. Due to superior performance, previous most works always prefer to apply small kernels instead of large kernels to handle time series sensor data for activity recognition. However, they do not intend to answer the key questions: why do large kernels underperform small kernels? How to close the performance gap? Intuitively, benefiting from larger receptive field (RF), larger kernels should have a great potential to model long-range dependencies in time series sensor data. So far, there has been little effort devoted to the larger-kernel design. In this article, we revisit the design of larger-kernel convolutions, which long have been neglected in the context of HAR. We find that both identity shortcut and structural re-parameterization can fully unleash the potential of larger-kernel convolutions. Extensive experiments and ablation studies on four mainstream benchmark datasets including PAMAP2, USC-HAD, UniMiB-SHAR, and OPPORTUNITY, show that our larger-kernel convolutions can further push the limit of small-kernel CNN performances under similar inference time, which can be used a drop-in replacement for small-kernel conv layers. For example, compared to the small-kernel baselines, our proposed approach can consistently boost recognition accuracy by 0.55%, 1.00%, 3.94%, and 1.64% on PAMAP2, USC-HAD, UniMiB-SHAR, and OPPORTUNITY, respectively, which is very competitive among the state-of-the-arts (SOTA). We believe that the incurred high performance is mainly due to larger effective RFs built via large kernels. The practical inference time is evaluated on a real hardware device. Our code can be available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MinghuiYao/ELK-HAR/</uri> .

Understanding the wheel–terrain interaction (WTI) is of great importance to improve the traversability of the planetary rovers. In this article, the design and implementation of an instrumented wheel (IW) for characterizing the WTIs are presented. Such an IW is the key technique of the wheel-on-limb detection (WOLD) system that aims for the forward terrain sensing and traversability prediction before the traversal of the rover. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">In situ</i> measurements of the WTI forces, wheel velocity, and wheel contact angle, as well as the wheel sinkage, can be achieved via such an IW. Experimental calibration is first implemented to guarantee the IW performance. Then, field tests in the representative environments are conducted to verify the measured parameters of WTIs. It has indicated that such a system concept would be potentially applicable in the future long-safe planetary traverses concerning the rover payloads development and functional validation.

Abstract Soft actuator is widely applied in soft grasp and artificial actuation. However, the characteristics of fast response and precise controllability are challenges for soft actuators. In this report, an ultrafast response and precisely controllable soft electromagnet actuator based on Ecoflex rubber film filled with neodymium-iron-boron is proposed. After optimizing the matrix and size of the actuator, the flexible magnetic actuator shows many merits including ultrafast response (&lt;0.1 s), large elongation (&lt;120%), excellent working stability, precision for long time working and customized designs. Based on customized patterns, different kinds of grippers have been made and they can realize the function of grasp and clutching of spherical and cylindrical objects. Furthermore, this actuator can be used on designed devices to build a pump system which can implement liquid injection and liquid blend. The proposed soft actuator has a great prospect in the practical application such as medical devices and soft robotics.

The neural radiance field (NeRF) is attracting increasing attentions from researchers in various fields. While NeRF has produced visually plausible results and found its potential applications in virtual reality, users are only allowed to rotate the camera to observe the scene represented as NeRF. We study the haptic interaction with NeRF models in this paper to enable the experience of touching objects reconstructed by NeRF. Existing haptic rendering algorithms do not work well for NeRF-represented models because NeRF is often noisy. We propose a stochastic haptic rendering method to deal with the collision response between the haptic proxy and NeRF. We validate our method with complex NeRF models and experimental results show the efficacy of our proposed algorithm.

Abstract Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.

Abstract The ability to detect both normal and tangential forces is essential for robotic tactile systems, while most available sensors implement this function at the cost of complex electrode structure and wiring. Here a fully flexible sensor for wireless detection of normal and tangential force based on resonant mechanism is reported. The components of the sensor are designed and optimized to meet the requirement, and the sensor shows good sensitivity, hysteresis characteristics, response time, and stability in the wired test. Wireless operation is carried out through inductive coupling, and the collected spectrum of reflection coefficient can reflect both normal force and tangential force information. In the case of individual input, the sensitivity is −0.19 MHz kPa −1 to normal force in the range of 0–200 kPa, and −0.031 dB kPa −1 to tangential force in the range of 0–50 kPa. As proofs of concept, maximum static friction coefficient tests and robotic gripping experiments are demonstrated for preliminary determination of surface roughness and material identification respectively. In particular, a total recognition accuracy of 92.5% is achieved by random forest‐based machine learning for balls with different materials in the gripping experiment.