Scalable order dispatching through Federated Multi-Agent Deep Reinforcement Learning

Efficient order dispatching is crucial for online ride-hailing systems, directly influencing user experience and platform revenue. Traditional methods often focus on maximizing immediate revenue through local observations of individual vehicles, ignoring the long-term potential benefits, the dynamic nature of dispatching systems, and the importance of collaboration among distributed vehicles. This typically results in suboptimal performance. To address these issues, we propose FedMARL4OD, a novel Federated Multi-Agent Deep Reinforcement Learning framework designed to optimize order dispatching. This framework integrates local learning via Multi-Agent Reinforcement Learning (MARL) for individual vehicles and global learning via Federated Multi-Agent Reinforcement Learning (FedMARL) across all vehicles. Specifically, we introduce an innovative reward mechanism in local learning that considers both the current revenue of each order and the supply–demand dynamics of the system related to potential future revenue, thereby improving dispatching performance. Moreover, we introduce a scalable model aggregation method in global learning that explicitly models interactions among distributed vehicles to facilitate collaborative learning. By progressively integrating local and global insights through average parameter aggregation, this method not only reduces communication overhead and enhances the learning efficiency of agents, but also ensures system scalability and maintains data privacy. Extensive real-world simulations demonstrate that FedMARL4OD outperforms baseline methods, achieving a 9.17% increase in Accumulated Driver Income (ADI) and a 7.75% improvement in Order Response Rate (ORR). The ADI improvements demonstrate the framework's effectiveness in boosting revenue, while the enhanced ORR indicates a quicker fulfillment of users’ requests, improving user experience.