Asynchronous Deep Double Dueling Q-learning for trading-signal execution in limit order book markets

Frontiers in Artificial Intelligence

Artificial Intelligence

Peer Nagy¹  Jan-Peter Calliess¹  Stefan Zohren² 

[1] Department of Engineering Science, Oxford-Man Institute of Quantitative Finance, University of Oxford, Oxford, United Kingdom;Department of Engineering Science, Oxford-Man Institute of Quantitative Finance, University of Oxford, Oxford, United Kingdom;Man Group, London, United Kingdom;Alan Turing Institute, London, United Kingdom;

关键词: limit order books;    quantitative finance;    reinforcement learning;    LOBSTER;    algorithmic trading;   

DOI  :  10.3389/frai.2023.1151003

 received in 2023-01-25, accepted in 2023-09-05,  发布年份 2023

来源: Frontiers

【 摘 要 】

We employ deep reinforcement learning (RL) to train an agent to successfully translate a high-frequency trading signal into a trading strategy that places individual limit orders. Based on the ABIDES limit order book simulator, we build a reinforcement learning OpenAI gym environment and utilize it to simulate a realistic trading environment for NASDAQ equities based on historic order book messages. To train a trading agent that learns to maximize its trading return in this environment, we use Deep Dueling Double Q-learning with the APEX (asynchronous prioritized experience replay) architecture. The agent observes the current limit order book state, its recent history, and a short-term directional forecast. To investigate the performance of RL for adaptive trading independently from a concrete forecasting algorithm, we study the performance of our approach utilizing synthetic alpha signals obtained by perturbing forward-looking returns with varying levels of noise. Here, we find that the RL agent learns an effective trading strategy for inventory management and order placing that outperforms a heuristic benchmark trading strategy having access to the same signal.

【 授权许可】

Unknown   
Copyright © 2023 Nagy, Calliess and Zohren.