Mean-shift outlier detection and filtering

PATTERN RECOGNITION

卷:115

Article

Yang, Jiawei^1,2  Rahardja, Susanto¹  Franti, Pasi^2,3 

[1] Northwestern Polytech Univ, Sch Marine Sci & Technol, Xian, Peoples R China

[2] Univ Eastern Finland, Sch Comp, Joensuu, Finland

[3] Shenzhen Technol Univ, Sch Big Data & Internet, Shenzhen, Guangdong, Peoples R China

关键词: Outlier detection;    Anomaly detection;    Mean-shift;    Medoid-shift;    Clustering;    Noise filtering;    Outlier filtering;   

DOI  :  10.1016/j.patcog.2021.107874

来源: Elsevier

【 摘 要 】

Traditional outlier detection methods create a model for data and then label as outliers for objects that deviate significantly from this model. However, when dat has many outliers, outliers also pollute the model. The model then becomes unreliable, thus rendering most outlier detectors to become ineffective. To solve this problem, we propose a mean-shift outlier detector. This detector employs a mean-shift technique to modify data and cancel the bias caused by the outliers. The mean-shift technique replaces every object by the mean of its k-nearest neighbors which essentially removes the effect of outliers before clustering without the need to know the outliers. In addition, it also detects outliers based on the distance shifted. Our experiments show that the proposed method works well regardless of the number of outliers in the data. This method outperforms all state-of-the-art methods tested, with both real-world numeric datasets as well as generated numeric and string datasets. (c) 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ )

【 授权许可】

Free