Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving

Lin Chi Wu, Zengjie Zhang, Sofie Haesaert, Zhiqiang Ma, Zhiyong Sun

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Reinforcement learning (RL) is an effective approach to motion planning in autonomous driving, where an optimal driving policy can be automatically learned using the interaction data with the environment. Nevertheless, the reward function for an RL agent, which is significant to its performance, is challenging to determine. The conventional work mainly focuses on rewarding safe driving states but does not incorporate the awareness of risky driving behaviors of the vehicles. In this paper, we investigate how to use risk-aware reward shaping to leverage the training and test performance of RL agents in autonomous driving. Based on the essential requirements that prescribe the safety specifications for general autonomous driving in practice, we propose additional reshaped reward terms that encourage exploration and penalize risky driving behaviors. A simulation study in OpenAI Gym indicates the advantage of risk-aware reward shaping for various RL agents. Also, we point out that proximal policy optimization (PPO) is likely to be the best RL method that works with risk-aware reward shaping.

Original languageEnglish
Title of host publicationIECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society
PublisherIEEE Computer Society
ISBN (Electronic)9798350331820
DOIs
StatePublished - 2023
Event49th Annual Conference of the IEEE Industrial Electronics Society, IECON 2023 - Singapore, Singapore
Duration: 16 Oct 202319 Oct 2023

Publication series

NameIECON Proceedings (Industrial Electronics Conference)
ISSN (Print)2162-4704
ISSN (Electronic)2577-1647

Conference

Conference49th Annual Conference of the IEEE Industrial Electronics Society, IECON 2023
Country/TerritorySingapore
CitySingapore
Period16/10/2319/10/23

Keywords

  • autonomous driving
  • motion planning
  • reinforcement learning
  • reward shaping
  • risk awareness

Fingerprint

Dive into the research topics of 'Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving'. Together they form a unique fingerprint.

Cite this