Southeast University — Publications

Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition

Article

The popularity of convolutional architecture has made sensor-based human activity recognition (HAR) become one primary beneficiary. By simply superimposing multiple convolution layers, the local features can be effectively captured from multi-channel time series sensor data, which could output high-performance activity prediction results. On the other hand, recent years have witnessed great success of Transformer model, which uses powerful self-attention mechanism to handle long-range sequence modeling tasks, hence avoiding the shortcoming of local feature representations caused by convolutional neural networks (CNNs). In this paper, we seek to combine the merits of CNN and Transformer to model multi-channel time series sensor data, which might provide compelling recognition performance with fewer parameters and FLOPs based on lightweight wearable devices. To this end, we propose a new Dual-branch Interactive Network (DIN) that inherits the advantages from both CNN and Transformer to handle multi-channel time series for HAR. Specifically, the proposed framework utilizes two-stream architecture to disentangle local and global features by performing conv-embedding and patch-embedding, where a co-attention mechanism is used to adaptively fuse global-to-local and local-to-global feature representations. We perform extensive experiments on three mainstream HAR benchmark datasets including PAMAP2, WISDM, and OPPORTUNITY, which verify that our method consistently outperforms several state-of-the-art baselines, reaching an F1-score of 92.05%, 98.17%, and 91.55% respectively with fewer parameters and FLOPs. In addition, the practical execution time is validated on an embedded Raspberry Pi P3 system, which demonstrates that our approach is adequately efficient for real-time HAR implementations and deserves as a better alternative in ubiquitous HAR computing scenario. Our model code will be released soon.

Yin Tang, Lei Zhang, Hao Wu et al. 2022

Publications

Computational design of planet regolith sampler based on Bayesian optimization

A Novel FTP+d Control Approach With Event- Driven Communication for Time-Delayed Bilateral Teleoperation Systems

Autonomous navigation for mobile robot based on a sonar ring and its implementation

A Control Strategy for Prosthetic Hand Based on Attention Concentration and EMG

A LiDAR SLAM Based on Clustering Features and Constraint Separation

Towards understanding hydrogen embrittlement under stray current interference through a quantitative method based on multifractal characteristics

Channel Attention for Sensor-Based Activity Recognition: Embedding Features into all Frequencies in DCT Domain

Discrimination and Memory Experiments on Haptic Perception of Softness

A Prediction Method of Contact Force in Precise Teleoperation with Time Delay

Haptic Texture Rendering of 2D Image Based on Adaptive Fractional Differential Method

A Miniature Multiaxis Force/Torque Sensor for Acupuncture

Novel Adaptive Finite-Time Control of Teleoperation System With Time-Varying Delays and Input Saturation

Continuous Shared Control for Robotic Arm Reaching Driven by a Hybrid Gaze-Brain Machine Interface

Interaction-Based Active Perception Method and Vibration-Audio-Visual Information Fusion for Asteroid Surface Material Identification

Distributed measurement system based on networked smart sensors with standardized interfaces

A quick physics-based deformation model and real-time force reflection algorithm

Implicit Neural Field Guidance for Teleoperated Robot-assisted Surgery

EEG-based investigation of haptic roles in multisensory emotion elicitation

Designer of a Multi-DOF Adaptive Finger for Prosthetic Hand

Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition