A hybrid neural network hidden Markov model approach for automatic story segmentation

Jia Yu, Lei Xie, Xiong Xiao, Eng Siong Chng

科研成果: 期刊稿件文章同行评审

13 引用 (Scopus)

摘要

We propose a hybrid neural network hidden Markov model (NN-HMM) approach for automatic story segmentation. A story is treated as an instance of an underlying topic (a hidden state) and words are generated from the distribution of the topic. The transition from one topic to another indicates a story boundary. Different from the traditional HMM approach, in which the emission probability of each state is calculated from a topic-dependent language model, we use deep neural network (DNN) to directly map the word distribution into topic posterior probabilities. DNN is known to be able to learn meaningful continuous features for words and hence has better discriminative and generalization capability than n-gram models. Specifically, we investigate three neural network structures: a feed-forward neural network, a recurrent neural network with long short-term memory cells (LSTM-RNN) and a modified LSTM-RNN with multi-task learning ability. Experimental results on the TDT2 corpus show that the proposed NN-HMM approach outperforms the traditional HMM approach significantly and achieves state-of-the-art performance in story segmentation.

源语言英语
页(从-至)925-936
页数12
期刊Journal of Ambient Intelligence and Humanized Computing
8
6
DOI
出版状态已出版 - 1 11月 2017

指纹

探究 'A hybrid neural network hidden Markov model approach for automatic story segmentation' 的科研主题。它们共同构成独一无二的指纹。

引用此