BA-GPT: Battlefield Awareness Interactive Q&A System Based on RAG

With the remarkable success of large language models (LLMs) such as ChatGPT, there is a growing recognition of the potential of generative AI trained on massive amounts of knowledge. Specially, early text-only LLMs had significant limitations when it came to situational awareness understanding in complex environments such as the battlefield. Therefore, the emergence of multimodal Large Language model(MLLMs) offers a new paradigm with opportunities and challenges for AI application. Current MLLMs face two dilemmas: “hallucination” undermines credibility, and parameterized knowledge is limited by training data, hindering real-time interaction. To address this in unmanned combat, we propose BA-GPT integrated with MLLMs to analyze and process multimodal data and execute tasks. It’s an interactive Q&A system for battlefield awareness based on RAG architecture, achieving real-time interaction with an external database. We also introduce retrieval-enhanced strategies to mitigate hallucination and enhance credibility, which improve awareness reasoning and prediction. Case studies show BA-GPT supports unmanned systems’ battlefield awareness.