Shing-Han Li
Skip Abstract Section
Abstract
Log management and log auditing have become increasingly crucial for enterprises in this era of information and technology explosion. The log analysis technique is useful for discovering possible problems in business processes and preventing illegal-intrusion attempts and data-tampering attacks. Because of the complexity of the dynamically changing environment, auditing a tremendous number of data is a challenging issue. We provide a real-time audit mechanism to improve the aforementioned problems in log auditing. This mechanism was developed based on the Lempel-Ziv-Welch (LZW) compression technique to facilitate effective compression and provide reliable auditing log entries. The mechanism can be used to predict unusual activities when compressing the log data according to pre-defined auditing rules. Auditors using real-time and continuous monitoring can perceive instantly the most likely anomalies or exceptions that could cause problems. We also designed a user interface that allows auditors to define the various compression and audit parameters, using real log cases in the experiment to verify the feasibility and effectiveness of this proposed audit mechanism. In summary, this mechanism changes the log access method and improves the efficiency of log analysis. This mechanism greatly simplifies auditing so that auditors must only trace the sources and causes of the problems related to the detected anomalies. This greatly reduces the processing time of analytical audit procedures and the manual checking time, and improves the log audit efficiency.
- S. A. Abu Taleb, H. M. J. Musafa, A. M. Khtoom, and K. Gharaybih. 2010. Improving LZW image compression. European Journal of Scientific Research 44, 3, 502--509.Google Scholar
- M. K. Ahmed, M. Hussain, and A. Raza. 2009. An automated user transparent approach to log Web URLs for forensic analysis. In Proceedings of the 5th International Conference on IT Security Incident Management and IT Forensics. 120--127. Google ScholarDigital Library
- S. Al-Fedaghi and B. Mattar. 2010. On security log management systems. Global Journal of Computer Science and Technology 10, 6, 73--82.Google Scholar
- S. Al-Fedaghi and F. Mahdi. 2010. Events classification in log audit. International Journal of Network Security & Its Applications (IJNSA) 2, 2, 58--73.Google ScholarCross Ref
- M. Al-Laham and M. M. El Emary. 2007. Comparative study between various algorithms of data compression techniques. International Journal of Computer Science and Network Security 7, 4, 281--291.Google Scholar
- M. G. Alles, A. Kogan, and M. A. Vasarhelyi. 2002. Feasibility and economics of continuous assurance. Auditing: A Journal of Practice & Theory 21, 1, 125--138.Google ScholarCross Ref
- M. G. Alles, F. Tostes, M. A. Vasarhelyi, and Riccio E. Luiz. 2006b. Continuous auditing: The USA experience and considerations for its implementation in Brazil. Journal of Information Systems and Technology Management 3, 2, 211--224.Google ScholarCross Ref
- M. G. Alles, G. Brennan, A. Kogan, and M. A. Vasarhelyi. 2006a. Continuous monitoring of business process controls: A pilot implementation of a continuous auditing system at Siemens. International Journal of Accounting Information Systems 7, 2, 137--161.Google ScholarCross Ref
- M. G. Alles, A. Kogan, and M. A. Vasarhelyi. 2008. Putting continuous auditing theory into practice: Lessons from two pilot implementations. Journal of Information Systems 22, 2, 195--214.Google ScholarCross Ref
- H. Altarawneh and M. Altarawneh. 2011. Data compression techniques on text files: A comparison study. International Journal of Computer Applications 26, 5, 42--54.Google ScholarCross Ref
- R. Baksa and M. Turoff. 2010. The current state of continuous auditing and emergency management's valuable contribution. In Proceedings of the 7th International ISCRAM Conference. 1--10.Google Scholar
- M. A. Bassiouni. 1985. Data compression in scientific and statistical databases. IEEE Transactions on Software Engineering 11, 10, 1047--1058. Google ScholarDigital Library
- F. Benford. 1938. The law of anomalous numbers. Proceedings of the American Philosophical Society 78, 4, 551--572.Google Scholar
- I. Brahmi, S. B. Yahia, H. Aouadi, and P. Poncelet. 2011. Towards a multiagent-based distributed intrusion detection system using data mining approaches. In Agents and Data Mining Interaction. Springer, Berlin Heidelberg, 173--194. Google ScholarDigital Library
- C. E. Brown, J. A. Wong, and A. A. Baldwin. 2006. Research streams in continuous audit: A review and analysis of the existing literature. In Proceedings of the 15th Annual Research Workshop on Artificial Intelligence and Emerging Technologies in Accounting, Auditing and Tax. 123--135.Google Scholar
- Canadian Institute of Chartered Accountants and American Institute of Certified Public Accountants (CICA/AICPA). 1999. Continuous Auditing, Research report. Toronto, Canada: CICA.Google Scholar
- C. Carslaw. 1988. Anomalies in income numbers: Evidence of goal oriented behavior. The Accounting Review (April) 321--327.Google Scholar
- M. V. Cerullo and M. J. Cerullo. 2003. Impact of SAS No. 94 on Computer Aided Audit Techniques. Information Systems Control Journal 1, 1, 53--57.Google Scholar
- D. Y. Chan and M. A. Vasarhelyi. 2011. Innovation and practice of continuous auditing. International Journal of Accounting Information Systems 12, 1, 152--160.Google ScholarCross Ref
- R. Debreceny, G. L. Gray, W. L. Tham, K. Y. Goh, and P. L. Tang. 2003. The development of embedded audit modules to support continuous monitoring in the electronic commerce environment. International Journal of Auditing 7, 2, 169--185.Google ScholarCross Ref
- C. Durtschi, W. Hillison, and C. Pacini. 2004. The effective use of benford's law to assist in detecting fraud in account data. Journal of Forensic Accounting 5, 17--34.Google Scholar
- S. Goedertier, J. De Weert, D. Martens, J. Vanthienen, and B. Baesens. 2011. Process discovery in event logs: An application in the telecom industry. Applied Soft Computing 11, 2, 1697--1710. Google ScholarDigital Library
- K. Govinda and Y. Kumar. 2012. Storage optimization in cloud environment using compression algorithm. International Journal of Emerging Trends & Technology in Computer Science (IJETTCS) 1, 1, 57--61.Google Scholar
- G. Greco, A. Guzzo, L. Pontieri, and D. Sacca. 2006. Discovering expressive process models by clustering log traces. IEEE Transactions on Knowledge and Data Engineering 18, 8, 1010--1027. Google ScholarDigital Library
- S. M. Groomer and U. S. Murthy. 1989. Continuous auditing of database applications: An embedded audit module approach. Journal of Information Systems 3, 2, 53--69.Google Scholar
- R. K. Gupta and R. Gupta. 2012. An evaluation of log file & compression mechanism. International Journal of Advanced Research in Computer and Communication Engineering 1, 2, 66--71.Google Scholar
- K. J. Han and J. C. Kieffer. 2007. The challenges of intrusion detection compression technology. In Proceedings of the Information Theory and Applications Workshop. 202--206.Google Scholar
- K. Handscombe. 2007. Continuous auditing from a practical perspective. Information Systems Control Journal 2, 51--55.Google Scholar
- S. Huang, D. Yen, L. Yang, and J. Hua. 2008. An investigation of Zipf's law for fraud detection. Decision Support Systems 46, 1, 70--83. Google ScholarDigital Library
- K. Kent and M. Souppaya. 2006. Guide to computer security log management. NIST Special Publication, 800--892.Google Scholar
- A. Kogan, E. F. Sudit, and M. A. Vasarhelyi. 1999. Continuous online auditing: A program of research. Journal of Information Systems 13, 2, 87--103.Google ScholarCross Ref
- K. R. Kolhe, P. R. Devale, and P. Shrivastava. 2010. High performance lossless multimedia data compression through improved dictionary. International Journal of Computer Applications 10, 1, 29--35.Google ScholarCross Ref
- S. Kumar, S. S. Bhadauria, and R. Gupta. 2012. A temporal database compression with differential method. International Journal of Computer Applications 4, 6, 65--68.Google ScholarCross Ref
- J. R. Kuhn and S. G. Sutton. 2006. Learning from worldcom: Implications for fraud detection through continuous assurance. Journal of Emerging Technologies in Accounting 3, 1, 61--80.Google ScholarCross Ref
- J. R. Kuhn and S. G. Sutton. 2010. Continuous auditing in erp system environments: The current state and future directions. Journal of Information Systems 24, 1, 91--113.Google ScholarCross Ref
- K. Leung and C. Leckie. 2005. Unsupervised anomaly detection in network intrusion detection using clusters. In Proceedings of the 28th Australasian Conference on Computer Science (ACSC’05). 38, 1, 333--342. Google ScholarDigital Library
- C. Lin, Y. Xie, and W. Wolf. 2004. LZQ-based code compression for VLIW embedded systems. In Proceedings of the Design, Automation and Test in Europe Conference. 76--81.Google Scholar
- M. Nigrini. 1996. A taxpayer compliance application of Benford's law. The Journal of the American Taxation Association 18, 1, 72--91.Google Scholar
- M. J. Nigrini and L. J. Mittermaier. 1997. The use of Benford's law as an aid in analytical procedures. Auditing: A Journal of Practice and Theory 16, 2, 52--67.Google Scholar
- P. M. Nishad and R. M. Chezian. 2012. A vital approach to compress the size of DNA sequence using LZW (Lempel-Ziv-Welch) with fixed length binary code and tree structure. International Journal of Computer Applications 43, 1, 7--9.Google ScholarCross Ref
- J. Redgrave, P. Ashish, F. Jason, H. Ted, and R. Charles. 2005. The SEDONA principles: Best practices recommendations & principles for addressing electronic document production. In Proceedings of the SEDONA Conference. 3--6.Google Scholar
- H. K. Reghbati. 1981. An overview of data compression techniques. IEEE Computer Magazine 14, 4, 71--75. Google ScholarDigital Library
- Z. Rezaee, R. Elam, and A. Sharbatoghlie. 2001. Continuous auditing: The audit of the future. Managerial Auditing Journal 16, 3, 150--158.Google ScholarCross Ref
- Z. Rezaee, A. Sharbatoghlie, R. Elam, and P. L. McMickle. 2002. Continuous auditing: Building automated auditing capability. Auditing 21, 1, 147--163.Google ScholarCross Ref
- M. A. Roth and S. J. Van Horn. 1993. Database compression. ACM SIGMOD Record 22, 3, 31--39. Google ScholarDigital Library
- S. Sarva. 2006. Continuous auditing through leveraging technology. ISACA Journal Online 2, 1, 1--4.Google Scholar
- G. Shields. 1998. Non-stop auditing. CA Magazine 131, 7, 39--40.Google Scholar
- N. K. Singh, D. S. Tomar, and B. N. Roy. 2010. An approach to understand the end user behavior through log analysis. International Journal of Computer Applications 5, 11, 27--34.Google ScholarCross Ref
- A. Singhal and S. Jajodia. 2006. Data warehousing and data mining techniques for intrusion detection systems. Distributed and Parallel Databases 20, 1, 149--166. Google ScholarDigital Library
- P. Skibiński and J. Swacha. 2007. Fast and efficient log file compression. In Proceedings of 11th East-European Conference on Advances in Databases and Information Systems (ADBIS). 330--342.Google Scholar
- P. Srinivasulu, D. Nagaraju, P. R. Kumar, and K. N. Rao. 2009. Classifying the network intrusion attacks using data mining classification methods and their performance comparison. International Journal of Computer Science and Network Security (IJCSNS) 9, 6, 11--18.Google Scholar
- K. R. Suneetha and R. Krishnamoorthi. 2009. Identifying user behavior by analyzing web server access log file. International Journal of Computer Science and Network Security (IJCSNS) 9, 4, 327--332.Google Scholar
- J. Thomas. 1989. Unusual patterns in reported earnings. The Accounting Review (October) 773--787.Google Scholar
- B. F. van Dongen and A. Adriansyah. 2009. Process mining: fuzzy clustering and performance visualization. In Business Process Management Workshops. Springer, Berlin Heidelberg, 158--169.Google Scholar
- W. M. P. Van der Aalst, M. H. Schonenberg, and M. Song. 2011. Time prediction based on process mining. Information Systems 36, 2, 450--475. Google ScholarDigital Library
- W. Van der Aalst. 2012. Process mining: Overview and opportunities. ACM Transactions on Management Information Systems (TMIS) 3, 2, 7. Google ScholarDigital Library
- M. A. Vasarhelyi and F. B. Halper. 1991. The continuous audit of online systems. Auditing: A Journal of Practice and Theory 10, 1, 110--125.Google Scholar
- M. A. Vasarhelyi and A. Kogan. 2004. Principles of analytic monitoring for continuous assurance. Journal of Emerging Technologies in Accounting 1, 1, 1--21.Google ScholarCross Ref
- L. Wen, J. Wang, W. M. P. van der Aalst, B. Huang, and J. Sun. 2007. A novel approach for process mining based on event types. Journal of Intelligent Information Systems 32, 2, 163--190. Google ScholarDigital Library
- T. A. Welch. 1984. A technique for high-performance data compression. IEEE Computer Journal 17, 6, 8--19. Google ScholarDigital Library
- A. Wespi, M. Dacier, and H. Debar. 2000. Intrusion detection using variable-length audit trail patterns. In Recent Advances in Intrusion Detection. Springer Berlin Heidelberg, 110--129. Google ScholarDigital Library
- L. White. 2005. Does internal control enhance or impede. Strategic Finance 86, 8, 6--7.Google Scholar
- M. H. F. Wilkinson. 1994. A simple data compression scheme for binary images of bacteria compared with commonly used image data compression schemes. Computer Methods and Programs in Biomedicine 42, 1, 255--262.Google ScholarCross Ref
- J. Ziv and A. Lempel. 1977. A universal algorithm for sequential data compression. IEEE Transactions on Information Theory 23, 3, 337--343. Google ScholarDigital Library
- J. Ziv and A. Lempel. 1978. Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory 24, 5, 530--536. Google ScholarDigital Library
Index Terms
- A Real-Time Audit Mechanism Based on the Compression Technique
Recommendations
A Compression Technique Based on Optimality of LZW Code (OLZW)
ICCCT '12: Proceedings of the 2012 Third International Conference on Computer and Communication TechnologyA loss less dictionary based data compression technique has been proposed in this paper which is based on the optimality of LZW code. The compression process is started with empty dictionary and if the next symbol to be encoded is already in dictionary, ...
A Hardware Architecture for the LZW Compression and Decompression Algorithms Based on Parallel Dictionaries
In this paper, a parallel dictionary based LZW algorithm called PDLZW algorithm and its hardware architecture for compression and decompression processors are proposed. In this architecture, instead of using a unique fixed-word-width dictionary a ...
Real-time lossless compression of mosaic video sequences
Special issue on multi-dimensional image processingThis paper presents a simple, fast coding technique for lossless compression of mosaic video data. The design of a video codec needs to strike a balance between the compression performance and the codec throughput. Aiming to make the encoding throughput ...
Comments