Regularized Estimation of High-dimensional Covariance Matrices.

High-dimensional;Covariance Matrix Estimation;Compressive Sensing;Recursive Group Lasso;Sparse Least-Mean-Square;Analog-to-Digital Converter;Electrical Engineering;Engineering;Electrical Engineering: Systems

Chen, YilunZhu, Ji ;

University of Michigan

关键词: High-dimensional;    Covariance Matrix Estimation;    Compressive Sensing;    Recursive Group Lasso;    Sparse Least-Mean-Square;    Analog-to-Digital Converter;    Electrical Engineering;    Engineering;    Electrical Engineering: Systems;   

Others  :  https://deepblue.lib.umich.edu/bitstream/handle/2027.42/86396/yilun_1.pdf?sequence=1&isAllowed=y

瑞士|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Many signal processing methods are fundamentally related to theestimation of covariance matrices. In cases where there are a largenumber of covariates the dimension of covariance matrices is muchlarger than the number of available data samples.This is especiallytrue in applications where data acquisition is constrained by limitedresources such as time, energy, storage and bandwidth.Thisdissertation attempts to develop necessary components for covarianceestimation in the high-dimensional setting.The dissertation makescontributions in two main areas of covariance estimation: (1) highdimensional shrinkage regularized covariance estimation and (2)recursive online complexity regularized estimation with applications ofanomaly detection, graph tracking,and compressive sensing.New shrinkage covariance estimation methods are proposed thatsignificantly outperform previous approaches in terms of mean squarederror.Two multivariate data scenarios are considered: (1)independently Gaussian distributed data; and (2) heavy tailedelliptically contoured data.For the former scenario we improve onthe Ledoit-Wolf (LW) shrinkage estimator using the principle ofRao-Blackwell conditioning and iterative approximation of theclairvoyant estimator. In the latter scenario, we apply a variancenormalizing transformation and propose an iterative robust LWshrinkage estimator that is distribution-free within the ellipticalfamily. The proposed robustified estimator is implemented via fixedpoint iterations with provable convergence and unique limit.A recursive online covariance estimator is proposed for trackingchanges in an underlying time-varying graphical model. Covarianceestimation is decomposed into multiple decoupled adaptive regressionproblems. A recursive recursive group lasso is derived using ahomotopy approach that generalizes online lasso methods to groupsparse system identification. By reducing the memory of the objectivefunction this leads to a group lasso regularized LMS that provablydominates standard LMS.Finally, we introduce a state-of-the-artsampling system, the Modulated Wideband Converter (MWC) which is basedon recently developed analog compressive sensing theory. By inferringthe block-sparse structures of the high-dimensional covariance matrixfrom a set of random projections, the MWC is capable of achievingsub-Nyquist sampling for multiband signals with arbitrary carrierfrequency over a wide bandwidth.