Log Event Filtering Using Clustering Techniques

log filtering;root cause analysis;Electrical and Computer Engineering

Wasfy, Ahmed

University of Waterloo

关键词: log filtering;    root cause analysis;    Electrical and Computer Engineering;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/4774/1/MASc-Wasfy.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

Large software systems are composed of various different run-time components, partnerapplications and, processes. When such systems operate they are monitored so that audits can beperformed once a failure occurs or when maintenance operations are performed. However, log filesare usually sizeable, and require filtering and reduction to be processed efficiently. Furthermore, thereis no apparent correspondence of how logged events relate to particular use cases the system may beperforming. In this thesis, we have developed a framework that is based on heuristic clusteringalgorithms to achieve log filtering, log reduction and, log interpretation. More specifically we definethe concept of the Event Dependency Graph, and we present event filtering and use caseidentification techniques, that are based on event clustering. The clustering process groups togetherall events that relate to a collection of initial significant events that relate to a use case. We refer tothese significant events as beacon events. Beacon events can be identified automatically or semiautomaticallyby examining log event types or event names against event types or event names in thecorresponding specification of a use case being considered (e.g. events in sequence diagrams).Furthermore, the user can select other or additional initial clustering conditions based on his or herdomain knowledge of the system. The clustering technique can be used in two possible ways. Thefirst is for large logs to be reduced or sliced, with respect to a particular use case so that, operators canbetter focus their attention to specific events that relate to specific operations. The second is for thedetermination of active use cases where operators select particular seed events of interest and thenexamine the resulting reduced logs against events or event types stemming from different alternativeknown use cases being considered, in order to identify the best match and consequently provideinsights on which of these alternative use cases may be running at any given time. The approach hasshown very promising results towards the identification of executing use cases among variousalternative ones in various runs of the Session Initiation Protocol.