Mitigating Jamming Attack in 5G Heterogeneous Networks: A Federated Deep Reinforcement Learning Approach

Jamming attack is one of the serious security breaches in the upcoming fifth-generation heterogeneous networks (5G HetNets). Most of the existing anti-jamming techniques, such as frequency hopping (FH) and direct sequence spread spectrum (DSSS) lack in self-adaptive capabilities to mitigate the security and privacy issues in highly dynamic 5G HetNet environment. In literature, although reinforcement learning (RL) has been explored a lot in designing various anti-jamming techniques to address the aforementioned problems, but these techniques suffer from the issue of large network resource consumption and slow convergence rate. To mitigate these issues, we propose a federated deep reinforcement learning (DRL) based anti-jamming technique for two-tier 5G HetNets. In the proposal, each femtocell of 5G HetNets is assumed to have multiple single antenna femto users (FUs) and a multi-antenna jammer used to jam the downlink signals from femto base station (FBS) to FUs. Aiming to improve the achievable rate at FUs in the presence of jammers, a joint optimization problem of beamforming and power allocation at FBSs is formulated by considering the quality-of-service (QoS) requirements of FUs. Due to the non-convex nature of the aforementioned optimization problem, we have used the Markov decision process (MDP) to transform the optimization problem into a multi-agent reinforcement learning (MARL) problem. Then, to solve this MDP with large number of states and action spaces, a federated deep reinforcement learning (DRL) scheme is proposed to maximize the achievable rate at FUs. The proposed scheme uses federated learning and dueling architecture of dueling double deep Q network (D3QN) to optimize the beamforming vectors and power allocation jointly at FBSs. The achievable rate performance of the proposed federated DRL scheme is compared with double deep Q network (DDQN) and deep Q network (DQN). Simulation results show that the proposed federated DRL scheme achieves 19.39% and 23.85% better achievable rate in comparison to DDQN and DQN schemes.