Multi-scale text Tiling for automatic story segmentation in Chinese broadcast news

Lei Xie, Jia Zeng, Wei Feng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

20 Scopus citations

Abstract

This paper applies Chinese subword representations, namely character and syllable n-grams, into the TextTiling-based automatic story segmentation of Chinese broadcast news. We show the robustness of Chinese subwords against speech recognition errors, out-of-vocabulary (OOV) words and versatility in word segmentation in lexical matching on errorful Chinese speech recognition transcripts. We propose a multi-scale TextTiling approach that integrates both the specificity of words and the robustness of subwords in lexical similarity measure for story boundary identification. Experiments on the TDT2 Mandarin corpus show that subword bigrams achieve the best performance among all scales with relative f-measure improvement of 8.84% (character bigram) and 7.11% (syllable bigram) over words. Multi-scale fusion of subword bigrams with words can bring further improvement. It is promising that the integration of syllable bigram with syllable sequence of word achieves an f-measure gain of 2.66% over the syllable bigram alone.

Original languageEnglish
Title of host publicationInformation Retrieval Technology - 4th Asia Information Retrieval Symposium, AIRS 2008, Revised Selected Papers
Pages345-355
Number of pages11
DOIs
StatePublished - 2008
Event4th Asia Information Retrieval Symposium, AIRS 2008 - Harbin, China
Duration: 15 Jan 200818 Jan 2008

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4993 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference4th Asia Information Retrieval Symposium, AIRS 2008
Country/TerritoryChina
CityHarbin
Period15/01/0818/01/08

Keywords

  • Multi-scale fusion
  • Multimedia retrieval
  • Spoken document retrieval
  • Spoken document segmentation
  • Story segmentation
  • TextTiling
  • Topic segmentation

Fingerprint

Dive into the research topics of 'Multi-scale text Tiling for automatic story segmentation in Chinese broadcast news'. Together they form a unique fingerprint.

Cite this