Discriminative Topic Segmentation of Text and Speech

13th International Conference on Artificial Intelligence and Statistics

计算机科学;数学科学

Mehryar Mohri Pedro Moreno Eugene Weinstein ; Google ; Google

PID  :  119799

来源: CEUR

【 摘 要 】

We explore automated discovery of topically coherent segments in speech or text sequences. We give two new discriminative topic segmen tation algorithms which employ a new measure of text similarity based on word cooccurrence. Both algorithms function by finding extrema in the similarity signal over the text, with the lat ter algorithm using a compact supportvector based description of a window of text or speech observations in word similarity space to over come noise introduced by speech recognition er rors and offtopic content. In experiments over speech and text news streams, we show that these algorithms outperform previous methods. We observe that topic segmentation of speech rec ognizer output is a more difficult problem than that of text streams; however, we demonstrate that by using a lattice of competing hypotheses rather than just the onebest hypothesis as input to the segmentation algorithm, the performance