TY - JOUR
T1 - Toward Automatic Market Making
T2 - An Imitative Reinforcement Learning Approach With Predictive Representation Learning
AU - Li, Siyuan
AU - Chen, Yafei
AU - Niu, Hui
AU - Zheng, Jiahao
AU - Lin, Zhouchi
AU - Li, Jian
AU - Guo, Jian
AU - Wang, Zhen
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025
Y1 - 2025
N2 - Market making (MM) is a crucial trading problem, where a market maker stands ready to buy and sell the asset at a publicly quoted price to provide market liquidity continuously. The primary challenges in market making include position risk, liquidity risk, and adverse selection. Emerging research works investigate applying reinforcement learning (RL) techniques to derive automatic MM strategies. However, existing methods mainly focus on addressing inventory risk using only single-level quotes, which restricts the trading flexibility. In this paper, we shed light on the optimization of market makers' returns under a smaller risk while ensuring market liquidity and depth. This paper proposes a novel RL-based market-making strategy Predictive and Imitative Market Making Agent (PIMMA). First, to ensure adequate liquidity, we design an action space to enable stably allocating orders of multi-level volumes and prices. Beyond that, we apply queue position information from these multi-price levels to encode them in the state representations. Second, aiming at alleviating adverse selection, we draw auxiliary signals into state representation and design a representation learning network structure to catch implicit information from the price-volume fluctuations. Finally, we develop a novel reward function to earn a fortune while avoiding holding a large inventory. With a provided expert demonstration, our method augments the RL objective with imitation learning and learns an effective MM policy. Experiments are conducted to evaluate the proposed method based on realistic historical data, and the results demonstrate PIMMA outperforms RL-based strategy in the perspectives of earning decent revenue and information by adopting the multi-risk aversion strategy.
AB - Market making (MM) is a crucial trading problem, where a market maker stands ready to buy and sell the asset at a publicly quoted price to provide market liquidity continuously. The primary challenges in market making include position risk, liquidity risk, and adverse selection. Emerging research works investigate applying reinforcement learning (RL) techniques to derive automatic MM strategies. However, existing methods mainly focus on addressing inventory risk using only single-level quotes, which restricts the trading flexibility. In this paper, we shed light on the optimization of market makers' returns under a smaller risk while ensuring market liquidity and depth. This paper proposes a novel RL-based market-making strategy Predictive and Imitative Market Making Agent (PIMMA). First, to ensure adequate liquidity, we design an action space to enable stably allocating orders of multi-level volumes and prices. Beyond that, we apply queue position information from these multi-price levels to encode them in the state representations. Second, aiming at alleviating adverse selection, we draw auxiliary signals into state representation and design a representation learning network structure to catch implicit information from the price-volume fluctuations. Finally, we develop a novel reward function to earn a fortune while avoiding holding a large inventory. With a provided expert demonstration, our method augments the RL objective with imitation learning and learns an effective MM policy. Experiments are conducted to evaluate the proposed method based on realistic historical data, and the results demonstrate PIMMA outperforms RL-based strategy in the perspectives of earning decent revenue and information by adopting the multi-risk aversion strategy.
KW - Market making
KW - deep reinforcement learning
KW - imitation learning
KW - predictive representation learning
UR - http://www.scopus.com/inward/record.url?scp=85205431450&partnerID=8YFLogxK
U2 - 10.1109/TETCI.2024.3451476
DO - 10.1109/TETCI.2024.3451476
M3 - 文章
AN - SCOPUS:85205431450
SN - 2471-285X
VL - 9
SP - 2427
EP - 2439
JO - IEEE Transactions on Emerging Topics in Computational Intelligence
JF - IEEE Transactions on Emerging Topics in Computational Intelligence
IS - 3
ER -