A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat

Kai Zhang; Yang Xu; Junjie Zhu

doi:10.1109/RICAI60863.2023.10489144

A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat

Kai Zhang, Yang Xu, Junjie Zhu

School of Civil Aviation

Northwestern Polytechnical University Xian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In order to adapt to the needs of underwater combat under low maneuverability conditions, this paper applies a soft actor-critic (SAC) based deep reinforcement learning algorithm to the strategy research of autonomous underwater vehicle (AUV) suicide attack. Firstly, the kinematic model of AUV is constructed and the underwater combat scenario and rules are established. Then the SAC algorithm is used in the strategy training for under-water combat environments, and the optimizations of reward functions are formulated for the characteristics of the environment. Finally, through the simulation and comparison with the DQN algorithm, the superiority of the proposed method is verified.

Original language	English
Title of host publication	2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	95-99
Number of pages	5
ISBN (Electronic)	9798350357950
DOIs	https://doi.org/10.1109/RICAI60863.2023.10489144
State	Published - 2023
Event	5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023 - Hangzhou, China Duration: 1 Dec 2023 → 3 Dec 2023

Publication series

Name	2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023

Conference

Conference	5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023
Country/Territory	China
City	Hangzhou
Period	1/12/23 → 3/12/23

Keywords

autonomous underwater vehicle
deep reinforcement learning
underwater combat

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/RICAI60863.2023.10489144

Cite this

Zhang, K., Xu, Y., & Zhu, J. (2023). A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat. In 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023 (pp. 95-99). (2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RICAI60863.2023.10489144

Zhang, Kai ; Xu, Yang ; Zhu, Junjie. / A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat. 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 95-99 (2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023).

@inproceedings{e446295e4194435195376f1ead821ab1,

title = "A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat",

abstract = "In order to adapt to the needs of underwater combat under low maneuverability conditions, this paper applies a soft actor-critic (SAC) based deep reinforcement learning algorithm to the strategy research of autonomous underwater vehicle (AUV) suicide attack. Firstly, the kinematic model of AUV is constructed and the underwater combat scenario and rules are established. Then the SAC algorithm is used in the strategy training for under-water combat environments, and the optimizations of reward functions are formulated for the characteristics of the environment. Finally, through the simulation and comparison with the DQN algorithm, the superiority of the proposed method is verified.",

keywords = "autonomous underwater vehicle, deep reinforcement learning, underwater combat",

author = "Kai Zhang and Yang Xu and Junjie Zhu",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023 ; Conference date: 01-12-2023 Through 03-12-2023",

year = "2023",

doi = "10.1109/RICAI60863.2023.10489144",

language = "英语",

series = "2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "95--99",

booktitle = "2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023",

}

Zhang, K, Xu, Y & Zhu, J 2023, A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat. in 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023. 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023, Institute of Electrical and Electronics Engineers Inc., pp. 95-99, 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023, Hangzhou, China, 1/12/23. https://doi.org/10.1109/RICAI60863.2023.10489144

A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat. / Zhang, Kai; Xu, Yang; Zhu, Junjie.
2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 95-99 (2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat

AU - Zhang, Kai

AU - Xu, Yang

AU - Zhu, Junjie

PY - 2023

Y1 - 2023

N2 - In order to adapt to the needs of underwater combat under low maneuverability conditions, this paper applies a soft actor-critic (SAC) based deep reinforcement learning algorithm to the strategy research of autonomous underwater vehicle (AUV) suicide attack. Firstly, the kinematic model of AUV is constructed and the underwater combat scenario and rules are established. Then the SAC algorithm is used in the strategy training for under-water combat environments, and the optimizations of reward functions are formulated for the characteristics of the environment. Finally, through the simulation and comparison with the DQN algorithm, the superiority of the proposed method is verified.

AB - In order to adapt to the needs of underwater combat under low maneuverability conditions, this paper applies a soft actor-critic (SAC) based deep reinforcement learning algorithm to the strategy research of autonomous underwater vehicle (AUV) suicide attack. Firstly, the kinematic model of AUV is constructed and the underwater combat scenario and rules are established. Then the SAC algorithm is used in the strategy training for under-water combat environments, and the optimizations of reward functions are formulated for the characteristics of the environment. Finally, through the simulation and comparison with the DQN algorithm, the superiority of the proposed method is verified.

KW - autonomous underwater vehicle

KW - deep reinforcement learning

KW - underwater combat

UR - http://www.scopus.com/inward/record.url?scp=85190947091&partnerID=8YFLogxK

U2 - 10.1109/RICAI60863.2023.10489144

DO - 10.1109/RICAI60863.2023.10489144

M3 - 会议稿件

AN - SCOPUS:85190947091

T3 - 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023

SP - 95

EP - 99

BT - 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023

Y2 - 1 December 2023 through 3 December 2023

ER -

Zhang K, Xu Y, Zhu J. A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat. In 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 95-99. (2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI 2023). doi: 10.1109/RICAI60863.2023.10489144

A SAC-Based Deep Reinforcement Learning Approach for Autonomous Underwater Vehicle Combat

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this