On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

JOURNAL OF MULTIVARIATE ANALYSIS

卷:101

Article

Biau, Gerard^1,2,3  Devroye, Luc⁴ 

[1] Univ Paris 06, LSTA, F-75252 Paris 05, France

[2] Univ Paris 06, LPMA, F-75252 Paris 05, France

[3] Ecole Normale Super, DMA, F-75230 Paris 05, France

[4] McGill Univ, Sch Comp Sci, Montreal, PQ H3A 2K6, Canada

关键词: Regression estimation;    Layered nearest neighbours;    One nearest neighbour estimate;    Bagging;    Random forests;   

DOI  :  10.1016/j.jmva.2010.06.019

来源: Elsevier

【 摘 要 】

Let X-1 X be identically distributed random vectors in R-d, independently drawn according to some probability density. An observation Xi is said to be a layered nearest neighbour (LNN) of a point x if the hyperrectangle defined by x and Xi contains no other data points. We first establish consistency results on L(x), the number of LNN of x. Then, given a sample (X, Y), (X-1, Y-1),, (X-n, Y-n) of independent identically distributed random vectors from Rd x R, one may estimate the regression function r(x) = E[Y] X = x by the LNN estimate r(n)(x), defined as an average over the Y's corresponding to those X, which are LNN of x. Under mild conditions on r, we establish the consistency of El r (x) r(x) towards 0 as n -> infinity, for almost all x and all p >= 1, and discuss the links between r and the random forest estimates of Breiman (2001) [8]. We finally show the universal consistency of the bagged (bootstrap-aggregated) nearest neighbour method for regression and classification. (c) 0 2010 Elsevier Inc. All rights reserved.

【 授权许可】

Free