科技报告详细信息
The Effects of Corpus Size and Homogeneity on Language Model
Rose, Tony ; Haddock, Nick
HP Development Company
关键词: speech recognition;    language modelling;    email;    dictation;   
RP-ID  :  HPL-97-70
学科分类:计算机科学(综合)
美国|英语
来源: HP Labs
PDF
【 摘 要 】

Generic speech recognition systems typically use language models that are trained to cope with a broad variety of input. However, many recognition applications are more constrained, often to a specific topic or domain. In cases such as these, a knowledge of the particular topic can be used to advantage. This report describes the development of a number of techniques for augmenting domain-specific language models with data from a more general source. Two investigations are discussed. The first concerns the problem of acquiring a suitable sample of the domain- specific language data from which to train the models. The issue here is essentially one of quality, since it is shown that not all domain-specific corpora are equal. Moreover, they can display significantly different characteristics that affect the quality of any language models built therefrom. These characteristics are defined using a number of statistical measures, and their significance for language modelling is discussed. The second investigation concerns the empirical development and evaluation of a set of language models for the task of email speech-to-text dictation. The issue here is essentially one of quantity, since it is shown that effective language models can be built from very modestly sized corpora, providing the training data matches the target application. Evaluations show that a language model trained on only 2 million words can perform better than one trained on a corpus of over 100 times that size.12 Pages

【 预 览 】
附件列表
Files Size Format View
RO201804100001344LZ 2421KB PDF download
  文献评价指标  
  下载次数:13次 浏览次数:67次