NOMA resource allocation method in IoV based on prioritized DQN-DDPG network

EURASIP Journal on Advances in Signal Processing

Mengli He¹  Xiaofei Wang¹  Zelong Liu¹  Yue Li¹ 

[1] Electronic Engineering School, Heilongjiang University, 150001, Harbin, China;

关键词: Prioritized deep Q network (Prioritized DQN);    Sum tree;    Importance sampling;    Deep deterministic policy gradient (DDPG);    Non-orthogonal multiple access (NOMA);   

DOI  :  10.1186/s13634-021-00828-1

来源: Springer

【 摘 要 】

To meet the demands of massive connections in the Internet-of-vehicle communications, non-orthogonal multiple access (NOMA) is utilized in the local wireless networks. In NOMA technique, various optimization methods have been proposed to provide optimal resource allocation, but they are limited by computational complexity. Recently, the deep reinforcement learning network is utilized for resource optimization in NOMA system, where a uniform sampled experience replay algorithm is used to reduce the correlation between samples. However, the uniform sampling ignores the importance of sample. To this point, this paper proposes a joint prioritized DQN user grouping and DDPG power allocation algorithm to maximize the system sum rate. At the user grouping stage, a prioritized sampling method based on TD-error (temporal-difference error) is proposed. At the power allocation stage, to deal with the problem that DQN cannot process continuous tasks and needs to quantify power into discrete form, a DDPG network is utilized. Simulation results show that the proposed algorithm with prioritized sampling can increase the learning rate and perform a more stable training process. Compared with the previous DQN algorithm, the proposed method improves the sum rate of the system by 2% and reaches 94% and 93% of the exhaustive search algorithm and optimal iterative power optimization algorithm, respectively. Although the sum rate is improved by only 2%, the computational complexity is reduced by 43% and 64% compared to the exhaustive search algorithm and the optimal iterative power optimization algorithm, respectively.

【 授权许可】

CC BY