Statistical inference in networks: fundamental limits and efficient algorithms

Community detection;Networks;Statistical inference

Xu, Jiaming

关键词: Community detection;    Networks;    Statistical inference;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/72799/Jiaming_Xu.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Today witnesses an explosion of data coming from various types of networks such as online social networks andbiological networks. The goal of this thesis is to understand when and how we can efficiently extract useful information from such network data.In the first part, we are interested in finding tight-knit communities within a network.Assuming the network is generated according to a planted cluster model, we derive a computationally efficient semidefinite programming relaxation of the maximum likelihood estimation methodand obtain a stronger performance guarantee than previously known.If the community sizes are linear in the total number of vertices, the guarantee matches up to a constant factor with the information limit which we also identify, and exactly matches without a constant gap when there is a single community or two equal-sized communities. However, if the community sizes are sublinear in the total number of vertices,the guarantee is far from the information limit. We conjecture that our algorithm achieves the computational limit below which nopolynomial-time algorithm can succeed. To provide evidence, we show that finding a communityin some regime below the conjectured computational limit but above the information limit is computationally intractable,assuming hardness of the well-known planted clique problem.The second part studies the problem of inferring the group preference for a set of itemsbased on the partial rankings over different subsets of the items provided by a group of users. A question of particular interest is how to optimally construct the graph used for assigning items to users for ranking. Assuming the partial rankings are generated independently according to the Plackett-Luce model, we analyzecomputationally efficient estimators based on maximum likelihood and rank-breaking schemes that decompose partial rankings into pairwise comparisons. We provide upper and lower bounds on the estimation error. The lower bounddepends on the degree sequence of the assignment graph, while the upper bound depends on the spectral gap of the assignment graph. When the graph is an expander, the lower and upper bounds match up to a logarithmic factor.The unifying theme for the two parts of the thesis is the spectral gap of the graph. In both cases, when the graph has a large spectral gap, accurate and efficient inference is possible via maximum likelihood estimation or its convex relaxation. However, when the spectral gap vanishes, accurate inference may be statisticallyimpossible, or it is statistically possible but may be computationally intractable.