Multilevel Class Token Transformer With Cross TokenMixer for Hyperspectral Images Classification

The transformer has become a prominent technique for hyperspectral image (HSI) classification, attributed to its capability to model global dependencies between features. Nevertheless, the predominant transformer-based methods rely on a direct information flow with a fixed number of tokens, causing the sequential transformer encoders to lack crucial interaction. This deficiency results in an inappropriate granularity of discriminative features and the loss of subtle patterns. In response to this limitation, we introduce a novel approach named Multi-level Class Token Transformer with Cross TokenMixer (MCTT) for HSI classification. Specifically, we explore a CNN stem network that incorporates 3D, 2D, and pointwise convolutions to encode local spatial-spectral information. The spectral-spatial features undergo transformation into semantic tokens using a semantic tokenizer. These tokens are then input into the transformer encoder to capture global interactions between different pixels. To create a hierarchical semantic representation, we propose a cross tokenmixer that integrates different levels of class tokens and patch tokens, enabling a multi-grained representation. The cross tokenmixers, with their varied number of tokens, facilitate the learning of distinct discriminative spectral-spatial representations and enable a comprehensive understanding of the HSI through a voting mechanism. Extensive experiments and ablation studies are conducted on three public HSI datasets to evaluate the performance of our proposed method. The results demonstrate the effectiveness and superior performance of our approach in HSI classification.