skip to main content
research-article

MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models (Extended Version)

Published:09 April 2019Publication History
Skip Abstract Section

Abstract

As Android has become increasingly popular, so has malware targeting it, thus motivating the research community to propose different detection techniques. However, the constant evolution of the Android ecosystem, and of malware itself, makes it hard to design robust tools that can operate for long periods of time without the need for modifications or costly re-training. Aiming to address this issue, we set to detect malware from a behavioral point of view, modeled as the sequence of abstracted API calls. We introduce MAMADROID, a static-analysis-based system that abstracts app’s API calls to their class, package, or family, and builds a model from their sequences obtained from the call graph of an app as Markov chains. This ensures that the model is more resilient to API changes and the features set is of manageable size. We evaluate MAMADROID using a dataset of 8.5K benign and 35.5K malicious apps collected over a period of 6 years, showing that it effectively detects malware (with up to 0.99 F-measure) and keeps its detection capabilities for long periods of time (up to 0.87 F-measure 2 years after training). We also show that MAMADROID remarkably overperforms DROIDAPIMINER, a state-of-the-art detection system that relies on the frequency of (raw) API calls. Aiming to assess whether MAMADROID’s effectiveness mainly stems from the API abstraction or from the sequencing modeling, we also evaluate a variant of it that uses frequency (instead of sequences), of abstracted API calls. We find that it is not as accurate, failing to capture maliciousness when trained on malware samples that include API calls that are equally or more frequently used by benign apps.

References

  1. Yousra Aafer, Wenliang Du, and Heng Yin. 2013. DroidAPIMiner: Mining API-level features for robust malware detection in Android. In SecureComm.Google ScholarGoogle Scholar
  2. Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In ACM Symposium on the Foundations of Software Engineering (FSE). Article 59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Panagiotis Andriotis, Martina Angela Sasse, and Gianluca Stringhini. 2016. Permissions snapshots: Assessing users’ adaptation to the Android runtime permission model. In IEEE Workshop on Information Forensics and Security (WIFS).Google ScholarGoogle ScholarCross RefCross Ref
  4. Daniel Arp, Michael Spreitzenbarth, Malte Hubner, Hugo Gascon, and Konrad Rieck. 2014. DREBIN: Effective and explainable detection of Android malware in your pocket. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle ScholarCross RefCross Ref
  5. Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. 2014. FlowDroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps. In ACM SIGPLAN Conference on Programming Language Design and Implementation. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Simon Bernard, Sébastien Adam, and Laurent Heutte. 2007. Using random forests for handwritten digit recognition. In Ninth International Conference on Document Analysis and Recognition (ICDAR). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Ravi Bhoraskar, Seungyeop Han, Jinseong Jeon, Tanzirul Azim, Shuo Chen, Jaeyeon Jung, Suman Nath, Rui Wang, and David Wetherall. 2014. Brahmastra: Driving apps to test the security of third-party components. In USENIX Security Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Iker Burguera, Urko Zurutuza, and Simin Nadjm-Tehrani. 2011. Crowdroid: Behavior-based malware detection system for Android. In ACM Workshop on Security and Privacy in Smartphones and Mobile Devices (SPSM). Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Gerardo Canfora, Eric Medvet, Francesco Mercaldo, and Corrado Aaron Visaggio. 2015. Detecting Android malware using sequences of system calls. In Workshop on Software Development Lifecycle for Mobile. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gerardo Canfora, Eric Medvet, Francesco Mercaldo, and Corrado Aaron Visaggio. 2016. Acquiring and analyzing app metrics for effective mobile malware detection. In IWSPA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Gerardo Canfora, Francesco Mercaldo, and Corrado Aaron Visaggio. 2016. An HMM and structural entropy based detector for Android malware: An empirical study. Computers 8 Security 61 (2016). Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Saurabh Chakradeo, Bradley Reaves, Patrick Traynor, and William Enck. 2013. MAST: Triage for market-scale mobile malware analysis. In ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec). Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Check Point. 2017. ExpensiveWall: A Dangerous ’Packed’ Malware on Google Play that Will Hit Your Wallet. https://blog.checkpoint.com/2017/09/14/expensivewall-dangerous-packed-malware-google-play-will-hit-wallet/.Google ScholarGoogle Scholar
  14. Check Point. 2017. FalseGuide misleads users on GooglePlay. https://blog.checkpoint.com/2017/04/24/falaseguide-misleads-users-googleplay/.Google ScholarGoogle Scholar
  15. Sen Chen, Minhui Xue, Lingling Fan, Shuang Hao, Lihua Xu, Haojin Zhu, and Bo Li. 2018. Automated poisoning attacks and defenses in malware detection systems: An adversarial machine learning approach. Computers 8 Security 73 (2018), 326--344.Google ScholarGoogle Scholar
  16. Sen Chen, Minhui Xue, Zhushou Tang, Lihua Xu, and Haojin Zhu. 2016. StormDroid: A streaminglized machine learning-based system for detecting Android malware. In AsiaCCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Yang Chen, Mo Ghorbanzadeh, Kevin Ma, Charles Clancy, and Robert McGwier. 2014. A hidden Markov model detection of malicious Android applications at runtime. In Wireless and Optical Communication Conference (WOCC).Google ScholarGoogle Scholar
  18. Jon Clay. 2016. Continued Rise in Mobile Threats for 2016. http://blog.trendmicro.com/continued-rise-in-mobile-threats-for-2016/.Google ScholarGoogle Scholar
  19. S. Dai, A. Tongaonkar, X. Wang, A. Nucci, and D. Song. 2013. NetworkProfiler: Towards automatic fingerprinting of Android apps. In IEEE INFOCOM.Google ScholarGoogle Scholar
  20. William Enck, Peter Gilbert, Seungyeop Han, Vasant Tendulkar, Byung-Gon Chun, Landon P. Cox, Jaeyeon Jung, Patrick McDaniel, and Anmol N. Sheth. 2014. TaintDroid: An information-flow tracking system for realtime privacy monitoring on smartphones. ACM Trans. Comput. Syst. 32, 2, Article 5 (2014). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. William Enck, Machigar Ongtang, and Patrick McDaniel. 2009. On lightweight mobile phone application certification. In ACM CCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Adrienne Porter Felt, Erika Chin, Steve Hanna, Dawn Song, and David Wagner. 2011. Android permissions demystified. In ACM CCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yu Feng, Osbert Bastani, Ruben Martins, Isil Dillig, and Saswat Anand. 2017. Automated synthesis of semantic malware signatures using maximum satisfiability. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle ScholarCross RefCross Ref
  24. Joshua Garcia, Mahmoud Hammad, Bahman Pedrood, Ali Bagheri-Khaligh, and Sam Malek. 2015. Obfuscation-resilient, Efficient, and Accurate Detection and Family Identification of Android Malware. Department of Computer Science, George Mason University, Tech. Rep (2015).Google ScholarGoogle Scholar
  25. Hugo Gascon, Fabian Yamaguchi, Daniel Arp, and Konrad Rieck. 2013. Structural detection of Android malware using embedded call graphs. In ACM Workshop on Artificial Intelligence and Security (AISec). Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Xi Ge, Kunal Taneja, Tao Xie, and Nikolai Tillmann. 2011. DyTa: Dynamic symbolic execution guided with static verification results. In International Conference on Software Engineering (ICSE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. SIGPLAN Not. 40, 6 (2005). Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Google. 2018. Android Security 2017 Year in Review. https://source.android.com/security/reports/Google_Android_Security_2017_Report_Final.pdf.Google ScholarGoogle Scholar
  29. Michael I. Gordon, Deokhwan Kim, Jeff H. Perkins, Limei Gilham, Nguyen Nguyen, and Martin C. Rinard. 2015. Information flow analysis of Android applications in DroidSafe. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle Scholar
  30. Michael Grace, Yajin Zhou, Qiang Zhang, Shihong Zou, and Xuxian Jiang. 2012. RiskRanker: Scalable and accurate zero-day Android malware detection. In International Conference on Mobile Systems, Applications, and Services (MobiSys). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Steven A. Hofmeyr, Stephanie Forrest, and Anil Somayaji. 1998. Intrusion detection using sequences of system calls. Journal of Computer Security 6, 3 (1998). Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Shifu Hou, Yanfang Ye, Yanggiu Song, and Melih Abdulhayoglu. 2017. HinDroid: An intelligent Android malware detection system based on structured heterogeneous information network. (2017).Google ScholarGoogle Scholar
  33. Yajin Zhou and Xuxian Jiang. 2013. Detecting passive content leaks and pollution in android applications. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle Scholar
  34. Ian Jolliffe. 2002. Principal Component Analysis. John Wiley 8 Sons, Ltd.Google ScholarGoogle Scholar
  35. Roberto Jordaney, Kumar Sharad, Santanu Kumar Dash, Zhi Wang, Davide Papini, Ilia Nouretdinov, and Lorenzo Cavallaro. 2017. Transcend: Detecting concept drift in malware classification models. In Proceedings of the 26th USENIX Security Symposium (USENIX Security 2017). Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. ElMouatez Billah Karbab, Mourad Debbabi, Abdelouahid Derhab, and Djedjiga Mouheb. 2018. MalDozer: Automatic framework for android malware detection using deep learning. Digital Investigation 24 (2018), S48--S59.Google ScholarGoogle ScholarCross RefCross Ref
  37. Michael J. Kearns. 1990. The Computational Complexity of Machine Learning. MIT press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jinyung Kim, Yongho Yoon, Kwangkeun Yi, Junbum Shin, and SWRD Center. 2012. ScanDal: Static analyzer for detecting privacy leaks in android applications. In MoST.Google ScholarGoogle Scholar
  39. William Klieber, Lori Flynn, Amar Bhosale, Limin Jia, and Lujo Bauer. 2014. Android taint flow analysis for app sets. In SOAP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Clemens Kolbitsch, Paolo Milani Comparetti, Christopher Kruegel, Engin Kirda, Xiao-yong Zhou, and XiaoFeng Wang. 2009. Effective and efficient malware detection at the end host. In USENIX Security Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Martina Lindorfer, Stamatis Volanis, Alessandro Sisto, Matthias Neugschwandtner, Elias Athanasopoulos, Federico Maggi, Christian Platzer, Stefano Zanero, and Sotiris Ioannidis. 2014. AndRadar: Fast discovery of Android applications in alternative markets. In Proceedings of the 11th Conference on Detection of Intrusions and Malware 8 Vulnerability Assessment (DIMVA).Google ScholarGoogle ScholarCross RefCross Ref
  42. Long Lu, Zhichun Li, Zhenyu Wu, Wenke Lee, and Guofei Jiang. 2012. CHEX: Statically vetting Android apps for component hijacking vulnerabilities. In ACM CCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for Android apps. In Joint Meeting on Foundations of Software Engineering (ESEC/FSE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Enrico Mariconti. 2019. TESSERACT’s evaluation framework and its use of MaMaDroid. https://www.benthamsgaze.org/2019/02/12/tesseracts-evaluation-framework-and-its-use-of-mamadroid/.Google ScholarGoogle Scholar
  45. Enrico Mariconti, Lucky Onwuzurike, Panagiotis Andriotis, Emiliano De Cristofaro, Gordon Ross, and Gianluca Stringhini. 2017. MaMaDroid: Detecting Android malware by building Markov chains of behavioral models. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle ScholarCross RefCross Ref
  46. Omid Mirzaei, Guillermo Suarez-Tangil, Juan Tapiador, and Jose M. de Fuentes. 2017. TriFlow: Triaging Android applications using speculative information flows. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, 640--651. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. David Morris. 2017. An Extremely Convincing WhatsApp Fake Was Downloaded More Than 1 Million Times From Google Play. http://fortune.com/2017/11/04/whatsapp-fake-google-play/.Google ScholarGoogle Scholar
  48. James R. Norris. 1998. Markov Chains. Cambridge University Press.Google ScholarGoogle Scholar
  49. Jon Oberheide and Charlie Miller. 2012. Dissecting the Android bouncer. In SummerCon.Google ScholarGoogle Scholar
  50. Damien Octeau, Somesh Jha, and Patrick McDaniel. 2012. Retargeting Android applications to Java bytecode. In ACM Symposium on the Foundations of Software Engineering (FSE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Lucky Onwuzurike, Mario Almeida, Enrico Mariconti, Jeremy Blackburn, Gianluca Stringhini, and Emiliano De Cristofaro. 2018. A family of Droids -- Android malware detection via behavioral modeling: Static vs. dynamic analysis. In Proceedings of the 16th IEEE Annual Conference on Privacy, Security and Trust (PST).Google ScholarGoogle ScholarCross RefCross Ref
  52. Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. 2018. TESSERACT: Eliminating experimental bias in malware classification across space and time. arXiv:1807.07838 (2018).Google ScholarGoogle Scholar
  53. Sebastian Poeplau, Yanick Fratantonio, Antonio Bianchi, Christopher Kruegel, and Giovanni Vigna. 2014. Execute this! Analyzing unsafe and malicious dynamic code loading in Android applications. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle Scholar
  54. Iasonas Polakis, Michalis Diamantaris, Thanasis Petsas, Federico Maggi, and Sotiris Ioannidis. 2015. Powerslave: Analyzing the energy consumption of mobile antivirus software. In DIMVA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Georgios Portokalidis, Philip Homburg, Kostas Anagnostakis, and Herbert Bos. 2010. Paranoid Android: Versatile protection for smartphones. In Annual Computer Security Applications Conference (ACSAC). Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Siegfried Rasthofer, Steven Arzt, and Eric Bodden. 2014. A machine-learning approach for classifying and categorizing Android sources and sinks. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle ScholarCross RefCross Ref
  57. Vaibhav Rastogi, Yan Chen, and Xuxian Jiang. 2013. DroidChameleon: Evaluating Android anti-malware against transformation attacks. In AsiaCCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Andrea Saracino, Daniele Sgandurra, Gianluca Dini, and Fabio Martinelli. 2016. Madam: Effective and efficient behavior-based android malware detection and prevention. IEEE Transactions on Dependable and Secure Computing (2016).Google ScholarGoogle Scholar
  59. Bhaskar Pratim Sarma, Ninghui Li, Chris Gates, Rahul Potharaju, Cristina Nita-Rotaru, and Ian Molloy. 2012. Android permissions: A perspective combining risks and benefits. In ACM Symposium on Access Control Models and Technologies. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Madhu K. Shankarapani, Subbu Ramamoorthy, Ram S. Movva, and Srinivas Mukkamala. 2011. Malware detection using assembly and API call sequences. Journal in Computer Virology 7, 2 (2011). Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. Kimberly Tam, Salahuddin J. Khan, Aristide Fattori, and Lorenzo Cavallaro. 2015. CopperDroid: Automatic reconstruction of Android malware behaviors. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle ScholarCross RefCross Ref
  62. May Ying Tee and Martin Zhang. 2018. Hidden App Malware Found on Google Play. https://www.symantec.com/blogs/threat-intelligence/hidden-app-malware-google-play.Google ScholarGoogle Scholar
  63. Peter Teufl, Michaela Ferk, Andreas Fitzek, Daniel Hein, Stefan Kraxberger, and Clemens Orthacker. 2016. Malware detection by applying knowledge discovery processes to application metadata on the Android Market (Google Play). Security and Communication Networks 9, 5 (2016), 389--419. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 1999. Soot - A Java bytecode optimization framework. In Conference of the Centre for Advanced Studies on Collaborative Research. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Dinesh Venkatesan. 2016. Android.Bankosy: All ears on voice call-based 2FA. http://www.symantec.com/connect/blogs/androidbankosy-all-ears-voice-call-based-2fa.Google ScholarGoogle Scholar
  66. Timothy Vidas and Nicolas Christin. 2014. Evading Android runtime analysis via sandbox detection. In AsiaCCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. Nicolas Viennot, Edward Garcia, and Jason Nieh. 2014. A measurement study of google play. ACM SIGMETRICS Performance Evaluation Review 42, 1 (2014). Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. Antonio Villas-Boas. 2018. More than 500,000 People Downloaded Games on the Google Play Store that were Infected with Nasty Malware -- Here are the 13 Apps Affected. https://www.businessinsider.com/google-play-store-game-apps-removed-malware-2018-11?r=US8IR=T.Google ScholarGoogle Scholar
  69. Michelle Y. Wong and David Lie. 2016. IntelliDroid: A targeted input generator for the dynamic analysis of Android malware. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle Scholar
  70. Ben Woods. 2016. Google Play has hundreds of Android apps that contain malware. http://www.trustedreviews.com/news/malware-apps-downloaded-google-play.Google ScholarGoogle Scholar
  71. Dong-Jie Wu, Ching-Hao Mao, Te-En Wei, Hahn-Ming Lee, and Kuo-Ping Wu. 2012. DroidMat: Android malware detection through manifest and API calls tracing. In Asia JCIS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Mingyuan Xia, Lu Gong, Yuanhao Lyu, Zhengwei Qi, and Xue Liu. 2015. Effective real-time Android application auditing. In IEEE Symposium on Security and Privacy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Lok Kwong Yan and Heng Yin. 2012. DroidScope: Seamlessly reconstructing the OS and Dalvik semantic views for dynamic Android malware analysis. In USENIX Security Symposium. Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Chao Yang, Zhaoyan Xu, Guofei Gu, Vinod Yegneswaran, and Phillip Porras. 2014. Droidminer: Automated mining and characterization of fine-grained malicious behaviors in Android applications. In ESORICS.Google ScholarGoogle Scholar
  75. Wei Yang, Xusheng Xiao, Benjamin Andow, Sihan Li, Tao Xie, and William Enck. 2015. AppContext: Differentiating malicious and benign mobile app behaviors using context. In International Conference on Software Engineering (ICSE). Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Zhemin Yang, Min Yang, Yuan Zhang, Guofei Gu, Peng Ning, and X. Sean Wang. 2013. AppIntent: Analyzing sensitive data transmission in Android for privacy leakage detection. In ACM CCS. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Hui Ye, Shaoyin Cheng, Lanbo Zhang, and Fan Jiang. 2013. DroidFuzzer: Fuzzing the Android apps with intent-filter tag. In International Conference on Advances in Mobile Computing and Multimedia (MoMM). Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Hanlin Zhang, Yevgeniy Cole, Linqiang Ge, Sixiao Wei, Wei Yu, Chao Lu, Genshe Chen, Dan Shen, Erik Blasch, and Khanh D. Pham. 2016. ScanMe mobile: A cloud-based Android malware analysis service. SIGAPP Appl. Comput. Rev. 16, 1 (2016). Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Nan Zhang, Kan Yuan, Muhammad Naveed, Xiaoyong Zhou, and XiaoFeng Wang. 2015. Leave me alone: App-level protection against runtime information gathering on Android. In IEEE Symposium on Security and Privacy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Yajin Zhou and Xuxian Jiang. 2012. Dissecting Android malware: Characterization and evolution. In IEEE Symposium on Security and Privacy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  81. Yajin Zhou, Zhi Wang, Wu Zhou, and Xuxian Jiang. 2012. Hey, you, get off of my market: Detecting malicious apps in official and alternative Android markets. In Annual Symposium on Network and Distributed System Security (NDSS).Google ScholarGoogle Scholar

Index Terms

  1. MaMaDroid: Detecting Android Malware by Building Markov Chains of Behavioral Models (Extended Version)

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Privacy and Security
      ACM Transactions on Privacy and Security  Volume 22, Issue 2
      May 2019
      214 pages
      ISSN:2471-2566
      EISSN:2471-2574
      DOI:10.1145/3316298
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 9 April 2019
      • Accepted: 1 February 2019
      • Revised: 1 November 2018
      • Received: 1 November 2017
      Published in tops Volume 22, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format