Adaptive dimension reduction for clustering high dimensional data

Ding, Chris ; He, Xiaofeng ; Zha, Hongyuan ; Simon, Horst

Lawrence Berkeley National Laboratory

关键词: Dna;    99 General And Miscellaneous//Mathematics, Computing, And Information Science;    Genes;    Implementation;    59 Basic Biological Sciences;   

DOI  :  10.2172/807420
RP-ID  :  LBNL--51472
RP-ID  :  AC03-76SF00098
RP-ID  :  807420

美国|英语

来源: UNT Digital Library

【 摘 要 】

It is well-known that for high dimensional data clustering, standard algorithms such as EM and the K-means are often trapped in local minimum. many initialization methods were proposed to tackle this problem, but with only limited success. In this paper they propose a new approach to resolve this problem by repeated dimension reductions such that K-means or EM are performed only in very low dimensions. Cluster membership is utilized as a bridge between the reduced dimensional sub-space and the original space, providing flexibility and ease of implementation. Clustering analysis performed on highly overlapped Gaussians, DNA gene expression profiles and internet newsgroups demonstrate the effectiveness of the proposed algorithm.