HADT: Human-AI Diagnostic Team via Hierarchical Reinforcement Learning

Medical online consultation is important to healthcare worldwide, with hundreds of millions of participants each year. However, expert-level online consultations are expensive due to the shortage of medical professionals, while AI models are unreliable because they have unpredictable risks. Therefore, we introduce human-machine collaboration to medical online consultation and focus on symptom inquiry, as the basis for disease diagnosis. There are two key issues: 1) how to design an intelligent assignment strategy that can determine whether doctors or models participate in each turn? 2) how to design an effective execution strategy that can improve the machine's inquiry ability among considerable symptoms? To address the above issues, we propose the Human-AI Diagnostic Team (HADT) framework based on Hierarchical Reinforcement Learning (HRL), which aims to achieve high accuracy with low manpower. Specifically, HADT has two layers. The upper one is responsible for assignment, in which we propose a module called master that enables intelligent human-machine assignments through the masked RL with reward shaping. The lower one is responsible for execution, consisting of a doctor and a proposed module called machine. This module can effectively ask about symptoms through the masked HRL with bottom-up training. Experiments on the public datasets show that HADT can achieve up to 89.4% accuracy with only 10.9% human effort, as confirmed by real clinical doctors using our online interface.