K-Means Clustering-based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition

Emotion is an important part of human interaction. Emotional recognition can greatly promote human-centered interaction techniques. On this basis, multimodal feature fusion can effectively improve the emotion recognition rate. However, in the multimodal feature fusion at the feature level, most of the methods do not consider the intrinsic relationship between different modes. Only the fusion of analysis and transformation of the feature matrices of different modes does not make better use of modal differences to improve the recognition rate. This problem led us to propose feature fusion method based on K-Means clustering and kernel canonical correlation analysis (KCCA). Clustering makes the classification of features not classified by mode, but by the degree of influence on emotional labels, thus positively affecting the results of KCCA. The experimental results obtained on the Savee database show that the proposed K-Means based KCCA improves overall classification performance and produces higher recognition rate than that of the state of art methods, such as the Informed Segmentation and Labeling Approach.