Multi-agent deep reinforcement learning algorithm with trend consistency regularization for portfolio management

Financial portfolio management is reallocating the asset into financial products, whose goal is to maximize the profit under a certain risk. Since AlphaGo debated human professional players, deep reinforcement learning (DRL) algorithm has been widely used in various fields, including quantitative trading. The multi-agent system is a relatively new research branch in DRL, and its performance is better than that of a single agent in most cases. In this paper, we propose a novel multi-agent deep reinforcement learning algorithm with trend consistency regularization (TC-MARL) to find the optimal portfolio. Here, we divide the trend of stocks of one portfolio into two categories and train two different agents to learn the optimal trading strategy under these two stock trends. First, we build a trend consistency (TC) factor to recognize the consistency of several stocks from one portfolio. When the trend of these stocks is consistent, the factor is defined as 1; the trend is inconsistent, the factor is defined as - 1. Based on it, a novel regularization related to the weights is proposed and added to the reward function, named TC regularization. And the TC factor value is used as the sign of the regularization term. In this way, two agents with different reward functions are constructed, which have the same policy model and value model. Afterward, the proposed TC-MARL algorithm will dynamically switch between the two trained agents to find the optimal portfolio strategy according to the market status. Extensive experimental results on the Chinese Stock Market show the effectiveness of the proposed algorithm.