skip to main content
research-article
Free Access

Advances and challenges in log analysis

Published:01 February 2012Publication History
Skip Abstract Section

Abstract

Logs contain a wealth of information to help manage systems.

References

  1. BlueGene/L Team. An overview of the BlueGene/L Supercomputer. IEEE Supercomputing and IBM Research Report (Nov. 2002). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Cantrill, B.M., Shapiro, M.W. and Leventhal, A.H. Dynamic instrumentation of production systems. Usenix 2004 Annual Technical Conference (Boston, MA, June 2004); http://www.usenix.org/event/usenix04/tech/general/full_papers/cantrill/cantrill.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Erlingsson, Ú., Peinado, M., Peter, S., Budiu and M. Fay: Extensible distributed tracing from kernels to clusters. In Proceedings of the 23 rd ACM Symposium on Operating Systems Principles, Cascais, Portugal (Oct. 2011); http://research.google.com/pubs/archive/37199.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Fonseca, R., Porter, G., Katz R., Shenker, S. and Stoica, I. X-Trace: A pervasive network-tracing framework. Usenix Symposium on Networked Systems Design and Implementation (Cambridge, MA, Apr. 2007). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Ganapathi, A., Chen, Y., Fox, A., Katz, R. H. and Patterson, D. A. Statistics-driven workload modeling for the cloud. Workshop on Self-Managing Database Systems at ICDE (2010), 87--92.Google ScholarGoogle ScholarCross RefCross Ref
  6. Ganapathi, A., Kuno, H. A., Dayal, U., Wiener, J. L., Fox, A., Jordan, M. I. and Patterson, D. A. Predicting multiple metrics for queries: Better decisions enabled by machine learning. International Conference on Data Engineering (2009) 592--603. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Gautam, A. and Stoica, I. ODR: output-deterministic replay for multicore debugging. ACM Symposium on Operating System Principles (2009), 193--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Nguyen, X., Huang, L. and Joseph, A. Support vector machines, data reduction, and approximate kernel matrices. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (2008), 137--153. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Oliner, A.J. and Aiken, A. Online detection of multi-component interactions in production systems. In Proceedings of the International Conference on Dependable Systems and Networks (Hong Kong, 2011); http://adam.oliner.net/files/oliner_dsn_2011.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Oliner, A.J., Kulkarni, A.V. and Aiken, A. Using correlated surprise to infer shared influence. In Proceedings of the International Conference on Dependable Systems and Networks (Chicago, IL, 2010), 191--200; http://adam.oliner.net/files/oliner_dsn_2010.pdf.Google ScholarGoogle ScholarCross RefCross Ref
  11. Rabkin, A. and Randy, K. Chukwa: A system for reliable large-scale log collection. USENIX Conference on Large Installation System Administration (2010), 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Sigelman, B., Barroso, L., Burrows, M., Stephenson, P., Plakal, M., Beaver, D., Jaspan, S. and Shanbhag, C. Dapper, a large-scale distributed systems tracing infrastructure. Google Technical Report; http://research.google.com/archive/papers/dapper-2010-1.pdf.Google ScholarGoogle Scholar
  13. Thrun, S. et al. Stanley: The robot that won the DARPA Grand Challenge. Journal of Field Robotics 23, 9 (2006), 661--692. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Xu, M. et al. A "flight data recorder" for enabling full-system multiprocessor deterministic replay. In Proceedings of the 30 th annual International Symposium on Computer Architecture (San Diego, CA, June 2003). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Xu, W., Huang, L., Fox, A., Patterson, D. and Jordan, M. Detecting large-scale system problems by mining console logs. In Proceeding of the 22 nd ACM Symposium on Operating Systems Principles (Big Sky, MT, Oct. 2009). Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yuan, D., Zheng, J., Park, S., Zhou, Y. and Savage, S. Improving software diagnosability via log enhancement. In Proceedings of Architectural Support for Programming Languages and Operating Systems (Newport Beach, CA, Mar. 2011); http://opera.ucsd.edu/paper/asplos11-logenhancer.pdf. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Advances and challenges in log analysis

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Communications of the ACM
        Communications of the ACM  Volume 55, Issue 2
        February 2012
        111 pages
        ISSN:0001-0782
        EISSN:1557-7317
        DOI:10.1145/2076450
        Issue’s Table of Contents

        Copyright © 2012 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 1 February 2012

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Popular
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format