A stochastic model of word occurrences in hierarchically structured written texts

SN Applied Sciences

Hiroshi Ogura¹  Yasutaka Hanada¹  Hiromi Amano¹  Masato Kondo¹ 

[1] Department of Information Science, Faculty of Arts and Sciences, Showa University;

关键词: Autocorrelation function;    Stochastic process;    Waiting time distribution;    Word occurrence;    Long-range correlation;   

DOI  :  10.1007/s42452-022-04953-w

来源: DOAJ

【 摘 要 】

Abstract In previous studies, we have treated real written texts as time series data and have tried to investigate dynamic correlations of word occurrences by utilizing autocorrelation functions (ACFs) and also by simulation of pseudo-text synthesis. The results showed that words that appear in written texts can be classified into two groups: a group of words showing dynamic correlations (Type-I words), and a group of words showing no dynamic correlations (Type-II words). In this study, we investigate the characteristics of these two types of words in terms of their waiting time distributions (WTDs) of word occurrences. The results for Type-II words show that the stochastic processes that govern generating Type-II words are superpositions of Poisson point processes with various rate constants. We further propose a model of WTDs for Type-I words in which the hierarchical structure of written texts is considered. The WTDs of Type-I words in real written texts agree well with the predictions of the proposed model, indicating that the hierarchical structure of written texts is important for generating long-range dynamic correlations of words.

【 授权许可】

Unknown