An Efficient Diverse-Branch Convolution Scheme for Sensor-Based Human Activity Recognition

Deep convolutional networks have recently achieved significant success in sensor-based human activity recognition (HAR). Existing works are mainly devoted to extracting multiscale activity features from sensor time series by increasing network depth or width, which are not friendly to mobile devices with limited computing resources. It still remains a challenging issue to strike an ideal trade-off between activity recognition performance and inference-time costs <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., the latency and memory footprint. To address this issue, we propose a diverse-branch convolution (DBC) scheme, which could strengthen the representation capacity of vanilla convolution via exploiting diverse branches of different scales and complexities to enrich activity feature space. Then DBC could be equivalently transformed into a single convolution layer after training for HAR deployment. In such manner, DBC only complicates training-time microstructure while preserving inference-time macrostructure, where the model performance can be boosted to a higher level and then converted back to its original inference-time structure for activity recognition. Extensive experiments are conducted on three publicly available benchmark datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., OPPORTUNITY, UniMiB-SHAR, WISDM, and a self-collected Weakly Labeled HAR dataset, which verify that our method can give a lift in performance while maintaining the same number of parameters and FLOPs as baselines. Finally, actual inference-time cost is evaluated for HAR deployment on a Raspberry Pi platform.