Deep Contextual Bandits Learning-Based Beam Selection for mmWave MIMO Systems

MmWave frequencies offer the potential for high data rates in wireless networks but are susceptible to blockage and scattering, making the beam selection in such systems a challenging problem to solve. In this paper, a deep contextual bandit (DCB) learning-based mmWave beam selection in mmWave MIMO wireless systems is proposed. Traditional beam selection methods rely on exhaustive search-based methods, which may not be feasible in fast-changing and highly dynamic environments. To address this challenge, the beam selection problem is modeled as a contextual bandit which is a natural tool for such an online decision-making problem. Due to the complexity of wireless channels in mmWave MIMO systems, neural network-based approaches are used to solve the problem of finding the best beam. In particular, neural ϵ-greedy and neural network committee for contextual bandits are used for beam selection where the input of the network is the raw channel data. Our simulation results demonstrate that the proposed approach results in near-optimal performance in terms of the achieved throughput within a reasonable training time. These results highlight the potential of deep contextual bandits for optimizing communication in 6G wireless systems.