Adaptive performance anomaly detection in distributed systems using online SVMs
Date
2020
Authors
Alvarez Cid-Fuentes, J.
Szabo, C.
Falkner, K.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
IEEE Transactions on Dependable and Secure Computing, 2020; 17(5):9281-941
Statement of Responsibility
Javier Alvarez Cid-Fuentes, Claudia Szabo, Katrina Falkner
Conference Name
Abstract
Performance anomaly detection is crucial for long running, large scale distributed systems. However, existing works focus on the detection of specific types of anomalies, rely on historical failure data, and cannot adapt to changes in system behavior at run time. In this work, we propose an adaptive framework for the detection and identification of complex anomalous behaviors, such as deadlocks and livelocks, in distributed systems without historical failure data. Our framework employs a two-step process involving two online SVM classifiers on periodically collected system metrics to identify at run time normal and anomalous behaviors such as deadlock, livelock, unwanted synchronization, and memory leaks. Our approach achieves over 0.70 F-score in detecting previously unseen anomalies and 0.78 F-score in identifying the type of known anomalies with a short delay after the anomalies appear, and with minimal expert intervention. Our experimental analysis uses system execution traces from our in-house distributed system with varied behaviors and a dataset by Yahoo!, and shows the benefits of our approach as well as future research challenges.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
© 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.