Subword latent semantic analysis for TextTiling-based automatic story segmentation of Chinese broadcast news

Yulian Yang, Lei Xie

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

11 Scopus citations

Abstract

This paper proposes to perform latent semantic analysis (LSA) on character/syllable n-gram sequences of automatic speech recognition (ASR) transcripts, namely subword LSA, as an extension of our previous work on subword TextTiling for automatic story segmentation of Chinese broadcast news. LSA represents the 'meaning' of a lexical term by a feature vector conveying the term's relations with other terms. We apply subword LSA vectors to the measurement of inter-sentence lexical score in TextTiling-based story segmentation. Subword n-grams are robust to speech recognition errors, especially out-of-vocabulary (OOV) words, in lexical matching on Chinese ASR transcripts. This work combines the concept matching merit of LSA and the robustness of subwords. Experimental results on the TDT2 Mandarin corpus show that subword-LSA-based TextTiling can effectively improve the story segmentation performance. Character-bigram-LSA-based TextTiling achieves the best F1-measure of 0.6598 with relative improvement of 17.4% over the conventional word-based TextTiling and 6.5% over our previous syllable-bigram-based TextTiling.

Original languageEnglish
Title of host publicationProceedings - 2008 6th International Symposium on Chinese Spoken Language Processing, ISCSLP 2008
Pages358-361
Number of pages4
DOIs
StatePublished - 2008
Event2008 6th International Symposium on Chinese Spoken Language Processing, ISCSLP 2008 - Kunming, China
Duration: 16 Dec 200819 Dec 2008

Publication series

NameProceedings - 2008 6th International Symposium on Chinese Spoken Language Processing, ISCSLP 2008

Conference

Conference2008 6th International Symposium on Chinese Spoken Language Processing, ISCSLP 2008
Country/TerritoryChina
CityKunming
Period16/12/0819/12/08

Keywords

  • Latent semantic analysis
  • Spoken document retrieval
  • Story segmentation
  • Subword
  • TextTiling
  • Topic segmentation

Fingerprint

Dive into the research topics of 'Subword latent semantic analysis for TextTiling-based automatic story segmentation of Chinese broadcast news'. Together they form a unique fingerprint.

Cite this