Deep Reinforcement Learning for Optimizing Order Book Imbalance-Based High-Frequency Trading Strategies

Abstract

High-frequency trading (HFT) exploits rapid execution speeds and market microstructure to seize short-lived opportunities. This paper proposes an innovative deep reinforcement learning (DRL) framework designed to enhance HFT strategies by leveraging order book imbalance as a predictive signal for short-term price dynamics. The approach integrates real-time order book data into a DRL model, enabling adaptive and optimized trading decisions in dynamic market conditions. The methodology involves extracting key features from order book snapshots, casting the trading problem as a Markov Decision Process (MDP), and employing a Deep Q-Network (DQN) to maximize long-term profitability. Experiments conducted on 12 months of high-frequency Apple Inc. (AAPL) stock data reveal the model's effectiveness, yielding a cumulative return of 15.2% and a Sharpe ratio of 1.8, surpassing traditional strategies like moving average crossover and market-making approaches. Robustness tests across diverse market scenarios further affirm its practical viability. This study advances the fusion of machine learning and financial trading by offering a scalable, data-driven solution for HFT optimization. Challenges such as data dependency and computational complexity are acknowledged, with future work suggested to incorporate transaction costs and explore advanced DRL architectures.