Skip to main content
Erschienen in: Automatic Control and Computer Sciences 7/2020

01.12.2020

On the Automatic Analysis of the Practical Resistance of Obfuscating Transformations

verfasst von: P. D. Borisov, Yu. V. Kosolapov

Erschienen in: Automatic Control and Computer Sciences | Ausgabe 7/2020

Einloggen, um Zugang zu erhalten

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

A method is developed for assessing the practical persistence of obfuscating transformations of programs. The method is based on the calculation of the similarity index for the original, obfuscated, and deobfuscated programs. Candidates are proposed for the similarity indices, which are based on such program characteristics as the control flow graph, symbolic execution time, and degree of coverage for symbolic execution. The control flow graph is considered the basis for building other candidates for program similarity indicators. On its basis, a new candidate is proposed for the similarity index, which, when calculated, finds the Hamming distances between the adjacency matrices of the control flow graphs of the compared programs. A scheme for estimating (analyzing) the persistence of obfuscating transformations is constructed. According to this scheme, the characteristics of the original, obfuscated, and deobfuscated programs are calculated and compared in accordance with the chosen comparison model. In particular, the developed scheme is suitable for comparing programs based on similarity indices. This paper develops and implements one of the key units of the constructed scheme, which is a block for obtaining the program characteristics compiled for the x86/x86_64 architecture. The developed unit allows finding the control flow graph, the time for symbolic execution, and the degree of coverage for symbolic execution. Selected results of operation of the constructed block are provided.
Literatur
1.
Zurück zum Zitat Siegmund, J., Program comprehension: Past, present, and future, IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 2016, vol. 5, pp. 13–20. Siegmund, J., Program comprehension: Past, present, and future, IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), 2016, vol. 5, pp. 13–20.
2.
Zurück zum Zitat Avidan, E. and Feitelson, D.G., From obfuscation to comprehension, Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension, 2015, pp. 178–181. Avidan, E. and Feitelson, D.G., From obfuscation to comprehension, Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension, 2015, pp. 178–181.
3.
Zurück zum Zitat Pozdeev, A.G., Krivopalov, V.N., Romashkin, E.V., and Radchenko, E.D., Mathematical and software tools for program obfuscation, Prikl. Diskretn. Mat., 2009, vol. 1, pp. 52–53. Pozdeev, A.G., Krivopalov, V.N., Romashkin, E.V., and Radchenko, E.D., Mathematical and software tools for program obfuscation, Prikl. Diskretn. Mat., 2009, vol. 1, pp. 52–53.
4.
Zurück zum Zitat Chernov, A.V., Analyzing confusing program transformations, 2002. http://www.citforum.ru/security/articles/analysis/. Chernov, A.V., Analyzing confusing program transformations, 2002. http://​www.​citforum.​ru/​security/​articles/​analysis/​.​
5.
Zurück zum Zitat Kuzurin, N., Shokurov, A., Varnovsky, N., and Zakharov, V., On the concept of software obfuscation in computer security, International Conference on Information Security, Berlin–Heidelberg: Springer, 2007, pp. 281–298. Kuzurin, N., Shokurov, A., Varnovsky, N., and Zakharov, V., On the concept of software obfuscation in computer security, International Conference on Information Security, Berlin–Heidelberg: Springer, 2007, pp. 281–298.
6.
Zurück zum Zitat Diffie, W. and Hellman, M., New directions in cryptography, IEEE Trans. Inf. Theory, 1976, vol. 22, no. 6, pp. 644–654.MathSciNetCrossRef Diffie, W. and Hellman, M., New directions in cryptography, IEEE Trans. Inf. Theory, 1976, vol. 22, no. 6, pp. 644–654.MathSciNetCrossRef
7.
Zurück zum Zitat Collberg, C.S. and Thomborson, C., Watermarking, tamper-proofing, and obfuscation tools for software protection, IEEE Trans. Software Eng., 2002, vol. 28, no. 8, pp. 735–746.CrossRef Collberg, C.S. and Thomborson, C., Watermarking, tamper-proofing, and obfuscation tools for software protection, IEEE Trans. Software Eng., 2002, vol. 28, no. 8, pp. 735–746.CrossRef
8.
Zurück zum Zitat Lee, B., Kim, Y., and Kim, J., binOb+: A framework for potent and stealthy binary obfuscation, Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security, ASIACCS 2010, 2010, pp. 271–281. Lee, B., Kim, Y., and Kim, J., binOb+: A framework for potent and stealthy binary obfuscation, Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security, ASIACCS 2010, 2010, pp. 271–281.
9.
Zurück zum Zitat Borello, J.M. and Me, L., Code obfuscation techniques for metamorphic viruses, J. Comput. Virol., 2008, vol. 4, no. 3, pp. 211–220.CrossRef Borello, J.M. and Me, L., Code obfuscation techniques for metamorphic viruses, J. Comput. Virol., 2008, vol. 4, no. 3, pp. 211–220.CrossRef
10.
Zurück zum Zitat Moser, A., Kruegel, C., and Kirda, E., Limits of static analysis for malware detection, Proceedings of Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), 2007, pp. 421–430. Moser, A., Kruegel, C., and Kirda, E., Limits of static analysis for malware detection, Proceedings of Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), 2007, pp. 421–430.
11.
Zurück zum Zitat Baiardi, F. and Sgandurra, D., An obfuscation-based approach against injection attacks, Proceedings of the Sixth International Conference on Availability, Reliability and Security (ARES), 2011, pp. 51–58. Baiardi, F. and Sgandurra, D., An obfuscation-based approach against injection attacks, Proceedings of the Sixth International Conference on Availability, Reliability and Security (ARES), 2011, pp. 51–58.
12.
Zurück zum Zitat Nurmukhametov, A.R., Use of diversifying and obfuscating transformations to change the signature of program code, Tr. Inst. Sist. Program. Ross. Akad. Nauk, 2016, vol. 28, no. 5, pp. 93–104. Nurmukhametov, A.R., Use of diversifying and obfuscating transformations to change the signature of program code, Tr. Inst. Sist. Program. Ross. Akad. Nauk, 2016, vol. 28, no. 5, pp. 93–104.
13.
Zurück zum Zitat Kosolapov, Y.V., About detection of code reuse attacks, Model. Anal. Inf. Sist., 2019, vol. 26, no. 2, pp. 213–228.CrossRef Kosolapov, Y.V., About detection of code reuse attacks, Model. Anal. Inf. Sist., 2019, vol. 26, no. 2, pp. 213–228.CrossRef
14.
Zurück zum Zitat Collberg, C., Thomborson, C., and Low, D., A Taxonomy of Obfuscating Transformations, Technical Report 148, The University of Auckland, 1997. Collberg, C., Thomborson, C., and Low, D., A Taxonomy of Obfuscating Transformations, Technical Report 148, The University of Auckland, 1997.
15.
Zurück zum Zitat Walenstein, A., El-Ramly, M., Cordy, J.R., Evans, W.S., Mahdavi, K., Pizka, M., Ramalingam, G., and von Gudenberg, J.W., Similarity in programs, Duplication, Redundancy, and Similarity in Software, 2007, pp. 1–8. Walenstein, A., El-Ramly, M., Cordy, J.R., Evans, W.S., Mahdavi, K., Pizka, M., Ramalingam, G., and von Gudenberg, J.W., Similarity in programs, Duplication, Redundancy, and Similarity in Software, 2007, pp. 1–8.
16.
Zurück zum Zitat Chipounov, V., Kuznetsov, V., and Candea, G., The S2E platform: Design, implementation, and applications, ACM Trans. Comput. Syst., 2012, vol. 30, no. 1, pp. 1–49.CrossRef Chipounov, V., Kuznetsov, V., and Candea, G., The S2E platform: Design, implementation, and applications, ACM Trans. Comput. Syst., 2012, vol. 30, no. 1, pp. 1–49.CrossRef
17.
Zurück zum Zitat Saudel, F. and Salwan, J., Triton: A dynamic symbolic execution framework, Symposium Sur La Security Des Technologies de L’information et Des Communications, SSTIC, 2015, pp. 31–54. Saudel, F. and Salwan, J., Triton: A dynamic symbolic execution framework, Symposium Sur La Security Des Technologies de L’information et Des Communications, SSTIC, 2015, pp. 31–54.
18.
Zurück zum Zitat Wang, Z., Ming, J., Jia, C., and Gao, D., Linear obfuscation to combat symbolic execution, Proceedings of Computer Security – ESORICS 2011, 2011, pp. 210–226. Wang, Z., Ming, J., Jia, C., and Gao, D., Linear obfuscation to combat symbolic execution, Proceedings of Computer Security – ESORICS 2011, 2011, pp. 210–226.
19.
Zurück zum Zitat Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., and Yin, H., Automatically identifying trigger-based behavior in malware, Botnet Detect., Adv. Inf. Secur., 2008, vol. 36, pp. 65–88.CrossRef Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., and Yin, H., Automatically identifying trigger-based behavior in malware, Botnet Detect., Adv. Inf. Secur., 2008, vol. 36, pp. 65–88.CrossRef
20.
21.
Zurück zum Zitat Cadar, C., Dunbar, D., and Engler, D.R., KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs, 8th USENIX Symposium on Operating Systems Design and Implementation, 2008, pp. 209–224. Cadar, C., Dunbar, D., and Engler, D.R., KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs, 8th USENIX Symposium on Operating Systems Design and Implementation, 2008, pp. 209–224.
22.
Zurück zum Zitat Shoshitaishvili, Y., et al., SoK: (State of) the art of war: Offensive techniques in binary analysis, IEEE Symposium on Security and Privacy, 2016, pp. 138–157. Shoshitaishvili, Y., et al., SoK: (State of) the art of war: Offensive techniques in binary analysis, IEEE Symposium on Security and Privacy, 2016, pp. 138–157.
23.
Zurück zum Zitat Sharif, M.I., Lanzi, A., Giffin, J.T., and Lee, W., Impeding malware analysis using conditional code obfuscation, Proceedings of NDSS, 2008, pp. 1–13. Sharif, M.I., Lanzi, A., Giffin, J.T., and Lee, W., Impeding malware analysis using conditional code obfuscation, Proceedings of NDSS, 2008, pp. 1–13.
24.
Zurück zum Zitat Udupa, S.K., Debray, S.K., and Madou, M., Deobfuscation: Reverse engineering obfuscated code, Proceedings of the 12th Working Conference on Reverse Engineering (WCRE'05), 2005, pp. 44–53. Udupa, S.K., Debray, S.K., and Madou, M., Deobfuscation: Reverse engineering obfuscated code, Proceedings of the 12th Working Conference on Reverse Engineering (WCRE'05), 2005, pp. 44–53.
25.
Zurück zum Zitat Nagarajan, V., Gupta, R., Zhang, X., Madou, M., and De Sutter, B., Matching control flow of program versions, IEEE International Conference on Software Maintenance, 2007, pp. 84–93. Nagarajan, V., Gupta, R., Zhang, X., Madou, M., and De Sutter, B., Matching control flow of program versions, IEEE International Conference on Software Maintenance, 2007, pp. 84–93.
26.
Zurück zum Zitat Bonfante, G., Kaczmarek, M., and Marion, J.Y., Control flow graphs as malware signatures, International Workshop on the Theory of Computer Viruses, 2007, pp. 1–6. Bonfante, G., Kaczmarek, M., and Marion, J.Y., Control flow graphs as malware signatures, International Workshop on the Theory of Computer Viruses, 2007, pp. 1–6.
27.
Zurück zum Zitat Park, Y., Reeves, D., Mulukutla, V., and Sundaravel, B., Fast malware classification by automated behavioral graph matching, Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research, 2010, pp. 1–4. Park, Y., Reeves, D., Mulukutla, V., and Sundaravel, B., Fast malware classification by automated behavioral graph matching, Proceedings of the Sixth Annual Workshop on Cyber Security and Information Intelligence Research, 2010, pp. 1–4.
28.
Zurück zum Zitat Kinable, J. and Kostakis, O., Malware classification based on call graph clustering, J. Comput. Virol., 2011, vol. 7, no. 4, pp. 233–245.CrossRef Kinable, J. and Kostakis, O., Malware classification based on call graph clustering, J. Comput. Virol., 2011, vol. 7, no. 4, pp. 233–245.CrossRef
29.
Zurück zum Zitat Lim, H.I., Comparing control flow graphs of binary programs through match propagation, IEEE 38th Annual Computer Software and Applications Conference, 2014, pp. 598–599. Lim, H.I., Comparing control flow graphs of binary programs through match propagation, IEEE 38th Annual Computer Software and Applications Conference, 2014, pp. 598–599.
30.
Zurück zum Zitat Dullien, T. and Rolles, R., Graph-Based Comparison of Executable Objects, 2005, pp. 1–8. Dullien, T. and Rolles, R., Graph-Based Comparison of Executable Objects, 2005, pp. 1–8.
31.
Zurück zum Zitat Chan, P.P.F. and Collberg, C., A method to evaluate CFG comparison algorithms, 14th International Conference on Quality Software, 2014, pp. 95–104. Chan, P.P.F. and Collberg, C., A method to evaluate CFG comparison algorithms, 14th International Conference on Quality Software, 2014, pp. 95–104.
32.
Zurück zum Zitat Axenovich, M., Kezdy, A., and Martin, R., On the editing distance of graphs, J. Graph Theory, 2008, vol. 58, no. 2, pp. 123–138.MathSciNetCrossRef Axenovich, M., Kezdy, A., and Martin, R., On the editing distance of graphs, J. Graph Theory, 2008, vol. 58, no. 2, pp. 123–138.MathSciNetCrossRef
33.
Zurück zum Zitat Borisov, P.D. and Kosolapov, Y.V., On the choice of characteristics for assessing the stability of obfuscating transformations, Sovremennye informatsionnye tekhnologii: Tendentsii i perspektivy razvitiya. Trudy XXV nauchnoi konferentsii SITO-2019 (Modern Information Technologies: Trends and Development Prospects. Proc. 25th Sci. Conf. SITO-2019), 2019, pp. 42–44. Borisov, P.D. and Kosolapov, Y.V., On the choice of characteristics for assessing the stability of obfuscating transformations, Sovremennye informatsionnye tekhnologii: Tendentsii i perspektivy razvitiya. Trudy XXV nauchnoi konferentsii SITO-2019 (Modern Information Technologies: Trends and Development Prospects. Proc. 25th Sci. Conf. SITO-2019), 2019, pp. 42–44.
34.
Zurück zum Zitat Lehman, M.M. and Belady, L.A., Program Evolution. Processes of Software Change, Academic Press, 1985. Lehman, M.M. and Belady, L.A., Program Evolution. Processes of Software Change, Academic Press, 1985.
35.
Zurück zum Zitat Schnappinger, M., Osman, M.H., Pretschner, A., Pizka, M., and Fietzke, A., Software quality assessment in practice: A hypothesis-driven framework, Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2018, pp. 1–6. Schnappinger, M., Osman, M.H., Pretschner, A., Pizka, M., and Fietzke, A., Software quality assessment in practice: A hypothesis-driven framework, Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2018, pp. 1–6.
36.
Zurück zum Zitat Borisov, P.D. and Kosolapov, Y.V., A model of the experimental analysis of the robustness of obfuscation algorithms, Sovremennye informatsionnye tekhnologii: Tendentsii i perspektivy razvitiya. Trudy XXV nauchnoi konferentsii SITO-2019 (Modern Information Technologies: Trends and Development Prospects. Proc. 25th Sci. Conf. SITO-2019), 2019, pp. 37–39. Borisov, P.D. and Kosolapov, Y.V., A model of the experimental analysis of the robustness of obfuscation algorithms, Sovremennye informatsionnye tekhnologii: Tendentsii i perspektivy razvitiya. Trudy XXV nauchnoi konferentsii SITO-2019 (Modern Information Technologies: Trends and Development Prospects. Proc. 25th Sci. Conf. SITO-2019), 2019, pp. 37–39.
37.
Zurück zum Zitat IDA Pro. https://www.hex-rays.com/products/ida/. IDA Pro. https://​www.​hex-rays.​com/​products/​ida/​.​
38.
Zurück zum Zitat The LLVM Compiler Infrastructure. https://llvm.org/. The LLVM Compiler Infrastructure. https://​llvm.​org/​.​
39.
Zurück zum Zitat McSema. https://github.com/trailofbits/mcsema. McSema. https://​github.​com/​trailofbits/​mcsema.​
40.
Zurück zum Zitat Junod, P., Rinaldini, J., Wehrli, J., and Michieliny, J., Obfuscator-LLVM – software protection for the masses, 2015 IEEE/ACM 1st International Workshop on Software Protection (SPRO), 2015, pp. 3–9. Junod, P., Rinaldini, J., Wehrli, J., and Michieliny, J., Obfuscator-LLVM – software protection for the masses, 2015 IEEE/ACM 1st International Workshop on Software Protection (SPRO), 2015, pp. 3–9.
Metadaten
Titel
On the Automatic Analysis of the Practical Resistance of Obfuscating Transformations
verfasst von
P. D. Borisov
Yu. V. Kosolapov
Publikationsdatum
01.12.2020
Verlag
Pleiades Publishing
Erschienen in
Automatic Control and Computer Sciences / Ausgabe 7/2020
Print ISSN: 0146-4116
Elektronische ISSN: 1558-108X
DOI
https://doi.org/10.3103/S0146411620070044

Weitere Artikel der Ausgabe 7/2020

Automatic Control and Computer Sciences 7/2020 Zur Ausgabe

Neuer Inhalt