Exploring Relevance and Coherence for Automated Text Scoring using Multi-task Learning

With the explosive growth of the information on the Internet, the evaluation of the quality and credibility of web content has become more important than ever before. In this work, we focus on the quality assessment of texts. Recently, various methods have been proposed for the automated text scoring task and obtained competitive results. However, few studies have focused on both relevance and coherence, which are two important factors in evaluating text quality. To improve the scoring task, we propose two auxiliary tasks using negative sampling and integrate them into a multi-task learning framework. The first auxiliary task is relevance modeling and the other one is coherence modeling. We evaluate our model on the Automated Student Assessment Prize (ASAP) dataset. Experimental results show that our model achieves higher Quadratic Weighted Kappa (QWK) scores with an improvement of 1.5% on average.