IMSBA: A Novel Integrated Sensing and Communication Beam Allocation Based on Multi-Agent Reinforcement Learning for mmWave Internet of Vehicles

In a multi-beam communication scenario where Infrastructure-to-Vehicle (I2V) and Vehicle-to-Vehicle (V2V) communications coexist, the limited spectrum of resources force V2V users to reuse the orthogonal frequency bands allocated to I2V, inevitably introducing cross-layer interference between I2V and V2V. Furthermore, the adoption of a multi-beam communication architecture exacerbates beam interference, significantly degrading the overall network’s communication and sensing performance. To address these challenges, this paper proposes an integrated sensing and communication (ISAC) beam allocation algorithm, termed IMSBA, which jointly optimizes beam direction, transmission power, and spectrum resource allocation to effectively mitigate the interference between I2V and V2V while maximizing the overall network performance. Specifically, IMSBA employs a joint optimization framework combining Multi-Agent Proximal Policy Optimization (MAPPO) with a Stackelberg game. Within this framework, MAPPO leverages vehicle perception data to dynamically optimize V2V beam steering and frequency selection, while the Stackelberg game reduces computational complexity through hierarchical decision-making and optimizes the joint power allocation among V2V users. Additionally, the proposed scheme incorporates a V2V cooperative sensing domain-sharing mechanism to enhance system robustness under adverse conditions. The experimental results demonstrated that, compared with existing baseline schemes, IMSBA achieved a 92.5% improvement in V2V energy efficiency while significantly enhancing both communication and sensing performance. This study provides an efficient and practical solution for spectrum-constrained scenarios in millimeter-wave Internet-of-Things (IoT), offering substantial theoretical insights and practical value for the efficient operation of intelligent transportation system (ITSs).