nach oben

The VLDB Journal

Erschienen in:

28.03.2024 | Regular Paper

Hilogx: noise-aware log-based anomaly detection with human feedback

verfasst von: Tong Jia, Ying Li, Yong Yang, Gang Huang

Erschienen in: The VLDB Journal | Ausgabe 3/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Log-based anomaly detection is essential for maintaining system reliability. Although existing log-based anomaly detection approaches perform well in certain experimental systems, they are ineffective in real-world industrial systems with noisy log data. This paper focuses on mitigating the impact of noisy log data. To this aim, we first conduct an empirical study on the system logs of four large-scale industrial software systems. Through the study, we find five typical noise patterns that are the root causes of unsatisfactory results of existing anomaly detection models. Based on the study, we propose HiLogx, a noise-aware log-based anomaly detection approach that integrates human knowledge to identify these noise patterns and further modify the anomaly detection model with human feedback. Experimental results on four large-scale industrial software systems and two open datasets show that our approach improves over 30% precision and 15% recall on average.

Vorheriger Artikel MM-DIRECT

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: The role of humans in interactive machine learning. Ai Magazine 35(4), 105–120 (2014)CrossRef

Brame, C.: Active learning. Vanderbilt University Center for Teaching (2016)

Das, S., Wong, W.K., Dietterich, T., Fern, A., Emmott, A.: Incorporating expert feedback into active anomaly discovery. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 853–858 (2016). https://doi.org/10.1109/ICDM.2016.0102

Das, S., Wong, W.K., Fern, A., Dietterich, T.G., Siddiqui, M.A.: Incorporating feedback into tree-based anomaly detection. arXiv preprint arXiv:1708.09441 (2017)

Du, M., Chen, Z., Liu, C., Oak, R., Song, D.: Lifelong anomaly detection through unlearning. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS ’19, p. 1283-1297. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3319535.3363226

Du, M., Li, F., Zheng, G., Srikumar, V.: Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS ’17, p. 1285–1298. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3133956.3134015

Fronza, I., Sillitti, A., Succi, G., Terho, M., Vlasenko, J.: Failure prediction based on log files using random indexing and support vector machines. J. Syst. Softw. 86(1), 2–11 (2013)CrossRef

Gomez Rodriguez, M., Leskovec, J., Schölkopf, B.: Structure and dynamics of information pathways in online media. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 23–32 (2013)

Görnitz, N., Kloft, M., Rieck, K., Brefeld, U.: Toward supervised anomaly detection. J. Artif. Intell. Res. 46, 235–262 (2013)MathSciNetCrossRef

10.

He, S., Zhu, J., He, P., Lyu, M.R.: Loghub: a large collection of system log datasets towards automated log analytics. arXiv preprint arXiv:2008.06448 (2020)

11.

Jia, T., Chen, P., Yang, L., Li, Y., Meng, F., Xu, J.: An approach for anomaly diagnosis based on hybrid graph model with logs for distributed services. In: 2017 IEEE International Conference on Web Services (ICWS), pp. 25–32 (2017). https://doi.org/10.1109/ICWS.2017.12

12.

Jia, T., Li, Y., Yang, Y., Huang, G., Wu, Z.: Augmenting log-based anomaly detection models to reduce false anomalies with human feedback. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3081–3089 (2022)

13.

Jia, T., Wu, Y., Hou, C., Li, Y.: Logflash: Real-time streaming anomaly detection and diagnosis from system logs for large-scale software systems. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), pp. 80–90 (2021). https://doi.org/10.1109/ISSRE52982.2021.00021

14.

Jia, T., Yang, L., Chen, P., Li, Y., Meng, F., Xu, J.: Logsed: Anomaly diagnosis through mining time-weighted control flow graph in logs. In: 2017 IEEE 10th International Conference on Cloud Computing (CLOUD), pp. 447–455 (2017). https://doi.org/10.1109/CLOUD.2017.64

15.

Kamar, E.: Directions in hybrid intelligence: complementing AI systems with human intelligence. In: IJCAI, pp. 4070–4073 (2016)

16.

Kamar, E., Hacker, S., Horvitz, E.: Combining human and machine intelligence in large-scale crowdsourcing. AAMAS 12, 467–474 (2012)

17.

Kim, J., Savchenko, V., Shin, K., Sorokin, K., Jeon, H., Pankratenko, G., Markov, S., Kim, C.J.: Automatic abnormal log detection by analyzing log history for providing debugging insight. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP ’20, p. 71–80. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3377813.3381371

18.

Lim, C., Singh, N., Yajnik, S.: A log mining approach to failure analysis of enterprise telephony systems. In: 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN), pp. 398–403 (2008). https://doi.org/10.1109/DSN.2008.4630109

19.

Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part V 13, pp. 740–755. Springer (2014)

20.

Meng, W., Liu, Y., Zhu, Y., Zhang, S., Pei, D., Liu, Y., Chen, Y., Zhang, R., Tao, S., Sun, P., et al.: Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In: IJCAI, vol. 19, pp. 4739–4745 (2019)

21.

Moulines, E., Bach, F.: Non-asymptotic analysis of stochastic approximation algorithms for machine learning. Adv. Neural Inf. Process. Syst. 24 (2011)

22.

Nandi, A., Mandal, A., Atreja, S., Dasgupta, G.B., Bhattacharya, S.: Anomaly detection using program control flow graph mining from execution logs. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, p. 215–224. Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2939672.2939712

23.

Pevnỳ, T.: Loda: Lightweight on-line detector of anomalies. Mach. Learn. 102(2), 275–304 (2016)MathSciNetCrossRef

24.

Reidemeister, T., Munawar, M.A., Ward, P.A.: Identifying symptoms of recurrent faults in log files of distributed information systems. In: 2010 IEEE Network Operations and Management Symposium—NOMS 2010, pp. 187–194 (2010). https://doi.org/10.1109/NOMS.2010.5488459

25.

Rodriguez, M.G., Balduzzi, D., Schölkopf, B.: Uncovering the temporal dynamics of diffusion networks. arXiv preprint arXiv:1105.0697 (2011)

26.

Siddiqui, M.A., Fern, A., Dietterich, T.G., Wright, R., Theriault, A., Archer, D.W.: Feedback-guided anomaly discovery via online optimization. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, p. 2200-2209. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3219819.3220083

27.

Tak, B.C., Tao, S., Yang, L., Zhu, C., Ruan, Y.: Logan: Problem diagnosis in the cloud using log-based reference models. In: 2016 IEEE International Conference on Cloud Engineering (IC2E), pp. 62–67 (2016). https://doi.org/10.1109/IC2E.2016.12

28.

Tong, J., Ying, L., Hongyan, T., Zhonghai, W.: An approach to pinpointing bug-induced failure in logs of open cloud platforms. In: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pp. 294–302 (2016). https://doi.org/10.1109/CLOUD.2016.0047

29.

Vaughan, J.W.: Making better use of the crowd: How crowdsourcing can advance machine learning research. J. Mach. Learn. Res. 18(1), 7026–7071 (2017)MathSciNet

30.

Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., Li, K.: Ai\(^{\wedge }\) 2: training a big data machine to defend. In: 2016 IEEE 2nd international conference on big data security on cloud (BigDataSecurity), IEEE international conference on high performance and smart computing (HPSC), and IEEE international conference on intelligent data and security (IDS), pp. 49–54. IEEE (2016)

31.

Xia, B., Bai, Y., Yin, J., Li, Y., Xu, J.: Loggan: a log-level generative adversarial network for anomaly detection using permutation event modeling. Inf. Syst. Front. 23, 285–298 (2021)CrossRef

32.

Xu, J., Chen, P., Yang, L., Meng, F., Wang, P.: Logdc: Problem diagnosis for declartively-deployed cloud applications with log. In: 2017 IEEE 14th International Conference on e-Business Engineering (ICEBE), pp. 282–287 (2017). https://doi.org/10.1109/ICEBE.2017.52

33.

Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.: Online system problem detection by mining patterns of console logs. In: 2009 Ninth IEEE International Conference on Data Mining, pp. 588–597 (2009). https://doi.org/10.1109/ICDM.2009.19

34.

Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP ’09, pp. 117–132. Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1629575.1629587

35.

Yang, L., Chen, J., Wang, Z., Wang, W., Jiang, J., Dong, X., Zhang, W.: Plelog: Semi-supervised log-based anomaly detection via probabilistic label estimation. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 230–231. IEEE (2021)

36.

Yang, Y., Wu, Y., Pattabiraman, K., Wang, L., Li, Y.: How far have we come in detecting anomalies in distributed systems? an empirical study with a statement-level fault injection method. In: 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), pp. 59–69 (2020). https://doi.org/10.1109/ISSRE5003.2020.00015

37.

Yen, T.F., Oprea, A., Onarlioglu, K., Leetham, T., Robertson, W., Juels, A., Kirda, E.: Beehive: Large-scale log analysis for detecting suspicious activity in enterprise networks. In: Proceedings of the 29th Annual Computer Security Applications Conference, ACSAC ’13, p. 199–208. Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2523649.2523670

38.

Yin, K., Yan, M., Xu, L., Xu, Z., Li, Z., Yang, D., Zhang, X.: Improving log-based anomaly detection with component-aware analysis. In: 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 667–671 (2020). https://doi.org/10.1109/ICSME46990.2020.00069

39.

Yu, X., Joshi, P., Xu, J., Jin, G., Zhang, H., Jiang, G.: Cloudseer: Workflow monitoring of cloud infrastructures via interleaved logs. SIGARCH Comput. Archit. News 44(2), 489–502 (2016). https://doi.org/10.1145/2980024.2872407

40.

Zhang, C., Peng, X., Sha, C., Zhang, K., Fu, Z., Wu, X., Lin, Q., Zhang, D.: Deeptralog: Trace-log combined microservice anomaly detection through graph-based deep learning. In: Proceedings of the 44th International Conference on Software Engineering, pp. 623–634 (2022)

41.

Zhang, X., Xu, Y., Lin, Q., Qiao, B., Zhang, H., Dang, Y., Xie, C., Yang, X., Cheng, Q., Li, Z., Chen, J., He, X., Yao, R., Lou, J.G., Chintalapati, M., Shen, F., Zhang, D.: Robust log-based anomaly detection on unstable log data. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2019, p. 807-817. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3338906.3338931

Titel: Hilogx: noise-aware log-based anomaly detection with human feedback
verfasst von: Tong Jia
Ying Li
Yong Yang
Gang Huang
Publikationsdatum: 28.03.2024
Verlag: Springer Berlin Heidelberg
Erschienen in: The VLDB Journal / Ausgabe 3/2024
Print ISSN: 1066-8888
Elektronische ISSN: 0949-877X
DOI: https://doi.org/10.1007/s00778-024-00843-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3/2024

Sub-trajectory clustering with deep reinforcement learning

Scalable decoupling graph neural network with feature-oriented optimization

Ingress: an automated incremental graph processing system

A new window Clause for SQL++

A new distributional treatment for time series anomaly detection

Label-constrained shortest path query processing on road networks