Secrecy Rate Maximization for THz-Enabled Femto Edge Users using Deep Reinforcement Learning in 6G

Dense deployment of femtocells in heterogeneous networks (HetNets) is critical for satisfying end-user quality-of-service (QoS) requirements. Femtocells can improve the network spectral efficiency and reduce user equipment’s power consumption. However, due to the distributed and dynamic nature, femto edge users (FEUs) cannot resist security attacks against the eavesdroppers. Hence, it is essential to enhance the secrecy rate of FEUs for secure and reliable data transmission. Aiming to improve the secrecy rate of FEUs, this paper formulates the power control problem of terahertz (THz)-enabled femto base station (FBS) consisting of multiple FEUs and an eavesdropper. Since the system is very complex and dynamic, addressing the non-convex optimization problem is difficult. First, we use the Markov decision process (MDP) to translate the optimization problem into a multi-agent deep reinforcement learning problem. Then, to solve the power control problem, we have used multi-agent Q-learning to enhance the learning ability and reduce the output dimension. Then, we have used deep deterministic policy gradient (DDPG) to convert the policy into deterministic one and to achieve efficient power control. Simulation results show that the proposed DRL based technique significantly improves the average secrecy rate of FEUs by 16.67% and 5% compared to the existing state-of-art schemes.