Secure Transmission Design for Cooperative NOMA in the Presence of Internal Eavesdropping

The application of successive interference cancellation (SIC) introduces critical security risks to cooperative non-orthogonal multiple access (NOMA) systems in the presence of untrustworthy network nodes, referred to as internal eavesdroppers. To address this potential security and reliability flaw, by assuming all users are untrusted, this letter investigates the effective secrecy throughput (EST) for a cooperative NOMA system, where a near user serves as an amplify-and-forward relay to help forward the information of a far user. Considering the inverse power allocation and SIC decoding order, a novel jamming strategy is proposed to enhance the security performance of the far user. Gauss-Chebyshev approximations of ESTs over Nakagami- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${m}$ </tex-math></inline-formula> channels are derived. Asymptotic EST expressions are proposed to provide further insights. Numerical results demonstrate that the proposed jamming strategy and the inverse power allocation and SIC decoding order are both essential for achieving positive secrecy rates for both users.