NLP Based on Prompt-Based Learning and Prompting Methods: A Survey

In this survey paper, we attempted to summarize the recent work of a paradigm shift in the natural processing language field that we call "Prompt-based learning". In recent years, the rapid development and stability of pre-trained language models have driven the advancement of this novel approach. Prompt-based learning leverages language models for clue-driven learning and has made significant strides in the field of natural language processing. To make predictions, the process begins with the design of text prompts based on the nature of the task and input-output format. Subsequently, user input data is combined with these text prompts to create a new text sequence, which is then fed into a pre-trained language model. Following this, information required for the task is extracted or interpreted from the generated text, and ultimately, the interpreted results are presented to the user or used for further processing, depending on the task requirements. However, there is currently no standardized framework for prompt-based learning. Existing code libraries for prompt-based learning are typically unregulated and provide limited functionality tailored to specific scenarios. Therefore, this paper aims to elucidate the theoretical foundations, various mathematical approaches, and existing research methodologies, as well as key aspects such as model selection, prompt optimization processes, and practical applications of prompt-based learning.